--- layout: post title: Recurrent neural networks tagline: " brief overview" permalink: /machine-learning/neural-network/recurrent_networks.html categories: [machine learning, neural networks] tags: [architecture, overview, recurrent network] ---
When applying machine learning to sequences, we often want to turn an input sequence into an output sequence that lives in a different domain.
Autoregressive models w t −1 w t −2 Predict the next term in a sequence from a fixed number of input(t-2) input(t-1) previous terms using “delay taps”.
Feed-forward neural nets These generalize autoregressive models by using one or more layers of non-linear hidden units. e.g. Bengio’s first language model. input(t) hidden input(t-2) input(t-1) input(t)
• If we give our generative model some hidden state, and if we give this hidden state its own internal dynamics, we get a much more interesting kind of model.
This inference is only tractable for two types of hidden state model.
time →
• These are generative models. They have a real-valued hidden state that cannot be observed directly.
• To predict the next output (so that we can shoot down the missile) we need to infer the hidden state.
time →
• Hidden Markov Models have a discrete one-of-N hidden state. Transitions between states are stochastic and controlled by a transition matrix. The outputs produced by a state are stochastic.
• To predict the next output we need to infer the probability distribution over hidden states.
Consider what happens when a hidden Markov model generates data.
• Consider the information that the first half of an utterance contains about the second half:
• All these aspects combined could be 100 bits of information that the first half of an utterance needs to convey to the second half. 2100
time →
• RNNs are very powerful, because they combine two properties:
• With enough neurons and time, RNNs can compute anything that can be computed by your computer.
• Linear dynamical systems and hidden Markov models are stochastic models.
Recurrent neural networks are deterministic.
What kinds of behaviour can RNNs exhibit?
But the computational power of RNNs makes them very hard to train.
In this video, I am going to give an overview of various types of models that have been used for sequences. I'll start with the simplest kinds of model, which is ultra aggressive models, that just try and predict the next term or the sequence from previous terms. I'll talk about more elaborate variants of them using hidden units.
And then I'll talk about, more interesting kinds of models, that have hidden state, and hidden dynamics. These include linear dynamical systems and hidden Markov models. Most of these are quite complicated kinds of models, and I don't expect you to understand all the details of them. The main point of mentioning them is to be able to show how recurrent your own networks are related to models of that kind.
When we're using machine learning to model sequences, we often want to turn one sequence into another sequence. For example, we might want to turn English words into French words or we might want to take a sequence of sound pressures and turn it into a sequence of word identities which is what's happening in speech recognition.
Sometimes we don't have a separate target sequence, and in that case we can get a teaching signal by trying to predict the next term in the input sequence. So the target output sequence is simply the input sequence with an advance of one time step. This seems much more natural, than trying to predict one pixel in an image from all the other pixels or one patch of an image from the rest of the image. One reason it probably seems more natural is that for temporal sequences, there is a natural order to do the predictions in.
Whereas for images it's not clear what you should predict from what. But in fact a similar approach works very well for images. When we predict the next term in a sequence, it blurs the distinction, between supervised and unsupervised learning, that I made at beginning of the course. So we use methods that were designed for supervised learning to predict the next term. But we don't require separate teaching signal. So in that sense, it's unsupervised.
I'm now going to give a quick review of some of the, other models of sequences, before we get on to using recurrent neural nets to model sequences. So a nice simple model for sequences that doesn't have any memory is an auto regressive model. What that does is take some previous terms in the sequence and try and predict the next term basically as a weighted average of previous terms. The previous terms might be individual values or they might be whole vectors.
And a linear auto regressive model would just take a weighted average of those to predict the next term. We can make that considerably more complicated by adding hidden units. So in a feedforward neural net, we might take some previous input terms, put them through some hidden units, and predict the next term. Memory list models are only one subclass of models that can be used for sequences. We can think about ways of generating sequences, and one very natural way to generate a sequence is to have a model that has some hidden state which has its own internal dynamics. So, the hidden state evolves according to its internal dynamics, and the hidden state also produces observations, and we get to see those observations.
That's a much more interesting kind of model. It can store information in its hidden state for a long time. Unlike the memoryless models, there's no simple band, to how far we have to look back before we can be sure it's not affecting things. If the dynamics of the hidden state is noisy and the way it generates outputs from its hidden state is noisy, then by observing the output of a generative model like this, you can never know for sure what it's hidden state was.
The best you can do is to infer probability distribution over the space of all possible hidden state vectors. You can know that it's probably in some part of the space and not another part of the space, but you can't pin it down exactly. So with a generative model like this, if you get to observe what it produces, and you now try to infer what the hidden state was, in general that's very hard, but there's two types of hidden state model for which the computation is tractable.
That is, there's a fairly straightforward computation that allows you to infer the probability distribution over the hidden state vectors that might have caused the data. Of course when we do this and apply it to real data. We're assuming that the real data is generated by our model. So that's typically what we do when we're modeling things. /We assume the data was generated by the model and then we infer what state the model must have been in, in order to generate that data./
The next three slides are mainly intended for people who already know about the two types of hidden state model I'm going to describe. The point of the slides is so that I make it clear how recurrent neural networks differ from those standard models. If you can't follow the details of the two standard models, don't worry too much. That's not the main point.
So one standard model is a linear dynamical system. It's very widely used in engineering. This is a generative model that has real valued hidden state. The hidden state has linear dynamics, shown by those red arrows on the right. And the dynamics has Gaussian noise, so that the hidden state evolves probabilistically.
There may also be driving inputs, shown at the bottom there, which directly influence the hidden state. So the inputs, influence the hidden state directly, the hidden state determines the output to predict the next output of a system like this, we need to be able to infer its hidden state. And these kinds of systems are used, for example, for tracking missiles.
In fact, one of the earliest uses of Gaussian distributions was for trying to track planets from noisy observations. Gaussian actually figured out that, if you assume Gaussian noise, you could do a good job of that. One nice property that a Gaussian has is that if you linearly transform a Gaussian you get another Gaussian. Because all the noise in a linear dynamic system is Gaussian. It turns out that the distribution over the hidden state given the observation so far, that is given the output so far, is also a Gaussian. It's a full covariance Gaussian, and it's quite complicated to compute what it is.
But it can be computed efficiently. And there's a technique called Kalman Filtering. This is an efficient recursive way of updating your representation of the hidden state given a new observation. So, to summarize, Given observations of the output of the system, we can't be sure what hidden state it was in, but we can, estimate a Gaussian distribution over the possible hidden states it might have been in.
Always assuming, of course, that our model is a correct model of the reality we're observing. A different kind of hidden state model that uses discrete distributions rather than Gaussian distributions, is a hidden Markov model. And because it's based on discrete mathematics, computer scientists love these ones. In a hidden Markov model, the hidden state consists of a one of N choice. So there a number of things called states. And the system is always in exactly one of those states. The transitions between states are probabilistic. They're controlled by a transition matrix which is simply a bunch of probabilities that say, if you're in state one at time one, What's the probability of you going to state three at time two? The output model is also stochastic.
So, the state that the system is in doesn't completely determine what output it produces. There's some variation in the output that each state can produce. Because of that, we can't be sure which state produced a given output. In a sense, the states are hidden behind this probabilistic veil, and that's why they're called hidden. Historically the reason hidden units in a neural network are called hidden, is because I like this term. It sounded mysterious, so I stole it from neural networks.
It is easy to represent the probability distribution across n states with n numbers. So, the nice thing about a hidden Markov model, is we can represent the probability distribution across its discreet states. So, even though we don't know what it, what state it's in for sure, we can easily represent the probability distribution. And to predict the next output from a hidden Markov model, we need to infer what hidden state it's probably in. And so we need to get our hands on that probability distribution. It turns out there's an easy method based on dynamic programming that allows us to take the observations we've made and from those compute the probability distribution across the hidden states.
Once we have that distribution, there is a nice elegant learning algorithm hidden Markov models, and that's what made them so appropriate for speech. And in the 1970s, they took over speech recognition. There's a fundamental limitation of HMMs. It's easiest to understand this limitation, if we consider what happens when a hidden Markov model generates data. At each time step when it's generating, it selects one of its hidden states. So if it's got N hidden states, the temporal information stored in the hidden state is at most log(N) n bits.
So that's all it knows about what it's done so far. So now let's consider how much information a hidden Markov model can convey to the second half of an utterance it produces from the first half. So imagine it's already produced the first half of an utterance. And now it's going to have to produce the second half. And remember, its memory of what it said for the first half is in which of the n states it's in. So its memory only has log(N) bits of information in it. To produce the second half that's compatible with the first half, we must make the syntax fit.
So for example, the number intend must agree. It also needs to make the semantics fit. It can't have the second half of the sentence be about something totally different from the first half. Also the intonation needs to fit so it would look very silly if the, intonation contour completely changed halfway through the sentence. There's a lot of other things that also have to fit. The accent of the speaker, The rate they're speaking at, How loudly they're speaking. And the vocal tract characteristics of the speaker.
All of those things must fit between the second half of the sentence and the first half. And so if you wanted a hidden Markov model to actually generate a sentence, the hidden state has to be able to convey all that information from the first half to the second half. Now the problem is that all of those aspects could easily come to a hundred bits of information. So the first half of the sentence needs to convey a hundred bits of information to the second half and that means that the hidden Markov model needs two to the hundreds states and that's just too many.
So that brings us to recurrence your own networks. They have a much more efficient way of remembering information. They're very powerful because they combine two properties that have distributed hidden state. That means, several different units can be active at once. So they can remember several different things at once. They don't just have one active unit. They're also nonlinear. You see, a linear dynamical system has a whole hidden state vector. So it's got more than one value at a time, but those values are constrained to act in a linear way so as to make inference easy, and in a recurrent neural network we allow the dynamics to be much more complicated.
With enough neurons and enough time, a recurring neuron network can compute anything that can be computed by your computer. It's a very powerful device. So linear dynamical systems and hidden Markov models are both stochastic models. That is the dynamics and the production of observations from the underlying state both involve intrinsic noise.
And the question is do models need to be like that. Well one thing to notice is that the posterior probability distribution over hidden states in either a limited anomical system or hidden Markov model is a deterministic function of the data that you've seen so far. That is the inference algorithm for these systems ends up with a probability distribution, and that probability distribution is just a bunch of numbers, and those numbers are a deterministic version of the data so far.
 And it might be a good idea to think of those numbers that constitute the hidden state of a recurrent neural network. They're very like the probability distribution for these simple stochastic models.
So what kinds of behavior can recurrent networks exhibit? Well, they can oscillate. That's obviously good for things like motion control, where when you're walking, for example, you want to know regular oscillation, which is your stride. They can settle to point attractors. That might be good for retrieving memories. And later on in the course we'll look at Hopfield nets where they use the settling to point attractors to store memories.
So the idea is you have a sort of rough idea of what you're trying to retrieve. You then let the system settle down to a stable point and those stable points correspond to the things you know about. And so by settling to that stable point you retrieve a memory. They can also behave chaotically if you set the weights in the appropriate regime. Often, chaotic behavior is bad for information processing, because in information processing, you want to be able to behave reliably if you want to achieve something.
There are some circumstances where it's a good idea. If you're up against a much smarter adversary, you probably can't outwit them, so it might be a good idea just to behave randomly. And one way to get the appearance of randomness is to behave chaotically. One nice thing about RNN's, which, a long time ago, I thought was gonna make them very powerful, is that an RNN could learn to implement lots of little programs, using different subsets of its hidden state. And each of these little programs could capture a nugget of knowledge. And all of these things could run in parallel, and interact with each other in complicated ways.
Unfortunately the computational power of recurrent neural networks makes them very hard to train. For many years, we couldn't exploit the computational power of recurrent neural networks. It was some heroic efforts. For example, Tony Robinson managed to make quite a good speech recognizer using recurrent nets. He had to do a lot of work implementing them on a parallel computer built out of transputers. And it was only recently that people managed to produce recurrent neural networks that outperformed Tony Robinson's
Figure 1: Recurrent Neural Network
Figure 2: Recurrent Neural Network in Unwrapped Representation
Assume that there is a time delay of 1 in using each connection. The recurrent net is just a layered net that keeps reusing the same weights.
It is easy to modify the backprop algorithm to incorporate linear constraints between the weights.
We compute the gradients as usual, and then modify the gradients so that they satisfy the constraints.
To constrain : w1 = w2 we need : Δw1 = Δw2 compute :
\(\frac{\partial{}E}{\partial{}w_{1}} and \frac{\partial{}E}{\partial{}w_{2}}\) use \(\frac{\partial{}E}{\partial{}w_{1}} + \frac{\partial{}E}{\partial{}w_{2}}\) for w1 and w2
We can think of the recurrent net as a layered, feed-forward net with shared weights and then train the feed-forward net with weight constraints. We can also think of this training algorithm in the time domain:
We need to specify the initial activity state of all the hidden and output units. We could just fix these initial states to have some default value like 0.5. But it is better to treat the initial states as learned parameters. We learn them in the same way as we learn the weights.
all the way to the initial states to get the gradient of the error function with respect to each initial state.
Figure 3: RNN input
We can specify inputs in several ways:
This is the natural way to model most sequential data.
Figure 4: Teaching signals
We can specify targets in several ways:
In this video, I'm going to talk about the back propagation through time algorithm. It's the standard way to train or recurrence your own network.
The algorithm is really quite simple once you have seen the equivalents between a recurrent neural network and a feed forward neural network that has one layer for each time step. I'll also talk about ways of providing input, and desired outputs, to recurrent neural networks. So the diagram shows a simple recurrent net with three interconnected neurons. We're going to assume there's a time delay of one in using each of those connections and that the network runs in discrete time, so the clock that has integer ticks. The key to understanding how to train a recurrent network is to see that a recurrent network is really just the same as a feed forward network, where you've expanded the recurrent network in time. So the recurrent network starts off in some initial state. Shown at the bottom there, times zero. And then uses the way some of these connections to get a new state, shown at time one. You then uses the same weights again to get another new state, and it uses the same weights again to get another new state and so on. So it's really just a lead feed forward network, where the weight is a constraint to be the same at every layer. Now backprop is good at learning when there are weight constraints. We saw this for convolutional nets and just to remind you, we can actually incorporate any linear constraint quite easily in backprop. So we compute the gradients as usual, as if the weights were not constrained. And then we modify the gradients, so that we maintain the constraints. So if we want W1 to equal W2, we start off with an equal and then we need to make sure that the changing W1 is equal to the changing W2. And we do that by simply taking the derivative of the area with respect to W1, the derivative with respect to W2, and adding or averaging them, and then applying the same quantity for updating both W1 and W2. So if the weights started off satisfying the constraints they'll continue to satisfy the constraints. The backpropagation through time algorithm is just the name for what happens when you think of a recurrent net as a lead feet forward net with shared weights, and you train it with backpropagation. So, we can think of that algorithm in the time domain. The forward pass builds up a stack of activities at each time slice. And the backward pass peels activities off that stack and computes error derivatives each time step backwards. That's why it's called back propagation through time. After the backward pass we can add together the derivatives at all the different time step for each particular weight. And then change all the copies of that weight by the same amount which is proportional to the sum or average of all those derivatives. There is an irritating extra issue. If we don't specify the initial state of the all the units, for example, if some of them are hidden or output units, then we have to start them off in some particular state. We could just fix those initial states to have some default value like 0.5, but that might make the system work not quite as well as it would otherwise work if it had some more sensible initial value. So we can actually learn the initial states. We treat them like parameters rather than activities and we learn them the same way as learned the weights. We start off with an initial random guess for the initial states. That is the initial states of all the units that aren't input units And then at the end of each training sequence we back propagate through time all the way back to the initial states. And that gives us the gradient of the error function with respects to the initial state. We then just, adjust the initial states by following, that gradient. We go downhill in the gradient, and that gives us new initial states that are slightly different. There's many ways in which we can provide the input to a recurrent neural net. We could, for example, specify the initial state of all the units. That's the most natural thing to do when we think of a recurrent net, like a feed forward net with constrained weights. We could specify the initial state of just a subset of the units or we can specify the states at every time stamp of the subset of the units and that's probably the most natural way to input sequential data. Similarly, there's many way we can specify targets for a recurrent network. When we think of it as feed forward network with constrained weights, the natural thing to do is to specify the desired final states for all of the units. If we're trying to train it to settle to some attractor, we might want to specify the desired states not just for the final time steps but for several time steps. That will cause it to actually settle down there, rather than passing through some state and going off somewhere else. So by specifying several states of the end, we can force it to learn attractors and it's quite easy as we back propagate to add in derivatives that we get from each time stamp. So the back propegation starts at the top, with the derivatives for the final time stamp. And then as we go back through the line before the top we add in the derivatives for that man, and so on. So it's really very little extra effort to have derivatives at many different layers. Or we could specify the design activity of a subset of units which we might think of as output units. And that's a very natural way to train a recurrent neural network that is meant to be providing a continuous output.
Figure 5: Binary addition
We can train a feedforward net to do binary addition, but there are obvious regularities that it cannot capture efficiently.
As a result, feedforward nets do not generalize well on the binary addition task.
Figure 6: Binary addition algorithm
This is a finite state automaton. It decides what transition to make by looking at the next column. It prints after making the transition. It moves from right to left over the two input numbers.
Figure 7: Addition in column
The network has two input units and one output unit.
It is given two input digits at each time step.
The desired output at each time step is the output for the column that was provided as input two time steps ago.
The 3 hidden units are fully interconnected in both directions.
The input units have feedforward connections that allow then to vote for the next hidden activity pattern. 3 fully interconnected hidden units
It learns four distinct patterns of activity for the 3 hidden units. These patterns correspond to the nodes in the finite state automaton.
A recurrent network can emulate a finite state automaton, but it is exponentially more powerful. With N hidden neurons it has 2N possible binary activity vectors (but only N2 weights)
In this video, I'm going to describe how a recurrent neural network solves a toy problem. It's a problem that's chosen to demonstrate what it is you can do with recurrent neural networks that you cannot do conveniently with feed-forward neural networks. The problem is adding up two binary numbers. Off to the recurrent neural network, has learned to solve the problem. It's interesting to look at its hidden states, and see how they relate to the hidden states in a finite state automaton that's solving the same problem.
So consider the problem of adding up two binary numbers. We could train a feed-forward neural network to do that. And the diagram on the right shows a network that gets some inputs and produces some outputs. But there's problems with using a feed-forward neural network. We have to decide in advance what the maximum number of digits is both for both of the input numbers and for the output number. And more importantly, the processing that we apply to the different bits of the input numbers, doesn't generalize.
That is, when we learn how to add up the last two digits and deal with the carries, that knowledges in some weights. And as we go to a different part of a long binary number, the knowledge will have to be in different weights. So we won't get automatic generalization.
As a result, although you can train a neuron feedfoward neural network, and it will eventually learn to do binary addition on fixed-length numbers, it's not an elegant way to solve the problem. This is a picture of the algorithm for binary addition. The states shown here are like the states in a hidden Markov model, except they're not really hidden. The system is in one state at a time. When it enters a state it performs an action. So it either prints a one or prints a zero and when it's in a state it gets some input, which is the two numbers in the next column. And that input causes it to go into a new state. So if you look on the top right, It's in the carry station and it's just printed a one. If it sees a one, one, it goes back in to the same stage and print another one. If however it sees a one, zero or zero, one, It goes into the carry state but prints a zero. If it sees a zero, zero, it goes into the no carry state, and prints a one. And so on.
So a recurring neuro-net for binary edition needs to have two input units and one output unit. It's given two input digits at each time stamp. And it also has to produce an output at each time step. And the output is the output for the column that it took in two time steps ago. The reason we need a delay of two time steps, is that it takes one time step to update the hidden units based on the inputs, and another time step to produce the output from the hidden state. So the net looks like this. I only gave it three hidden units.
That's sufficient to do the job. It would learn faster with more hidden units, but it can do it with three. The three hidden units are fully interconnected and they have connections in both directions that don't necessarily have the same weight. In fact in general they don't have the same weight. The connections between hidden units allow the pattern of one time step to insensate the pattern of the next time step. The input units have feed forward connections to the hidden units and that's how it sees the two digits in a column.
And similarly, the hidden units have feed forward connections to the output unit and that's how it produces its output. It's interesting to look at what the recurring neural network learns. It learns four distinct patterns of activity in its three hidden units. And these patterns correspond to the nodes in the finite state automaton for binary addition.
We must confuse the units in a neural network, with the nodes in a final state automaton. The nodes in the finite state automaton correspond to the activity vectors of the recurrent neural network. The automaton is restricted to being exactly one state at each time. And similarly, the hidden units are restricted to have exactly one activity vector at each time in the recurrent neural network. So a recurrent neural network can emulate a finite state automaton but it's exponentially more powerful in its representation. With any hidden neurons, it has 2N to the N possible binary activity vectors.
Of course it only has n squared weights so it can't necessarily make full use of all that representational power. But if the bottleneck is in the representation a recurrent neural network can do much better than a finite state automaton. This is important when the input stream has two separate things going on at once. A finite state automaton needs to square its number of states in order to deal with the fact that there's two things going on at once. A recurrent neural network only needs to double its number of hidden units. By doubling the number of units, it does of course square the number of binary vector states that it has.
Figure 8: RNN backward pass
There is a big difference between the forward and backward passes. In the forward pass we use squashing functions (like the logistic) to prevent the activity vectors from exploding. The backward pass, is completely linear. If you double the error derivatives at the final layer, all the error derivatives will double.
What happens to the magnitude of the gradients as we back-propagate through many layers?
Typical feed-forward neural nets can cope with these exponential effects because they only have a few hidden layers.
In an RNN trained on long sequences (e.g. 100 time steps) the gradients can easily explode or vanish.
Even with good initial weights, its very hard to detect that the current target output depends on an input from many time-steps ago.
If we start a trajectory within an attractor, small changes in where we start make no difference to where we end up. But if we start almost exactly on the boundary, tiny changes can make a huge difference.
Make the RNN out of little modules that are designed to remember values for a long time.
Deal with the vanishing gradients problem by using a fancy optimizer that can detect directions with a tiny gradient but even smaller curvature.
Initialize the input → hidden and hidden → hidden and output → hidden connections very carefully so that the hidden state has a huge reservoir of weakly coupled oscillators which can be selectively driven by the input.
Initialize like in Echo State Networks, but then learn all of the connections using momentum.
In this video, I'm going to talk about the exploding and vanishing gradients problem, which is what makes it difficult to train recurrent neural networks.
For many years, researchers in RNN networks thought they would never be able to train these networks to model dependencies over long time periods. But at the end of this video, I can describe four different ways in which that can now be done.
To understand why it's so difficult to train recurrent neural networks, we have to understand a very important difference between the forward and backward passes in a recurrent neural net. In the forward pass, we used squashing functions, like the logistic, to prevent the activity vectors from exploding. So, if you look at the picture on the right, each neuron is using a logistic unit shown by that blue curve and it can't output any value greater than one or less than zero, So that stops explosions.
The backward pass, however, is completely linear. Most people find this very surprising. If you double the error derivatives is it the final layers of this net, all the error derivatives will double when you back propagate. So, if you look at the red dots that I put on the blue curves, We'll suppose those are the activity levels of the neurons on the forward pass.
And so, when you back propagate, you're using the gradients of the blue curves at those red dots. So the red lines are meant to throw the tangents to the blue curves at the red dots. And, once you finish the forward pass, the slope of that tangent is fixed. You now back propagate and the back propagation is like going forwards though a linear system in which the slope of the non-linearity has been fixed.
Of course, each time you back propagate, the slopes will be different because they were determined by the forward pass. But during the back propagation, it's a linear system and so it suffers from a problem of linear systems, which is when you iterate, they tend to either explode or die. So when we backpropagate through many layers if the weights are small the gradients will shrink and become exponentially small.
And that means that when you backpropagate through time gradients that are many steps earlier than the targets arrive will be tiny. Similarly, if the weights are big, the gradients will explode. And that means when you back propagate through time, the gradients will get huge and wipe out all your knowledge. In a feed-forward neural net, unless it's very deep, these problems aren't nearly as bad because we typically only have a few hidden layers. But as soon as we have a recurrent neural network trained on a long sequence, for example 100 time steps, then if the gradients are growing as we back propagate, we'll get whatever that growth rate is to the power of 100 and if they're dying, we'll get whatever that decay is to the power of 100 and, so, they'll either explode or vanish.
We might be able to initialize the weights very carefully to avoid this and more recent work, shows that indeed careful initialization of the weights does make things work much better. But even with good initial weights, it's hard to detect the dependency of the current output or an input from many time-steps ago. So it's hard to make the output depend on things that happened a long time ago.
RNN's have difficulty dealing with long-range dependencies. Here's an example of exploding and dying gradients for a system that's trying to learn attractors. So suppose we try and train a recurrent neural network, So that whatever state we started in, it ends up in one of these two attractor states. So we're going a learn blue basin of attraction and a pink basin of attraction. And if we start anywhere within the blue basin of attraction, we will end up at the same point.
What that means is that, small differences in our initial state make no difference to where we end up. So the derivative of the final state with respect to changes in the initial state, is zero. That's vanishing gradients. When we back propagate through the dynamics of this system we will discover there's no gradients from where you start, and the same with the pink basin of attraction.
If however, we start very close to the boundary between the two attractors. Then, a tiny difference in where we start, that's the other side of the watershed, makes the huge difference to where we end up, that's the explosion gradient problem.
And so whenever your trying to use a recurrent neural network to learn attractors like this, you're bound to get vanishing or exploding gradients. It turns out, there's at least four effective ways to learn a recurrent neural network. The first is a method called long short term memory and I'll talk about that more in this lecture.
The idea is we actually change the architecture of the neural network to make it good at remembering things. The second method is to use a much better optimizer that can deal with very small gradients. I'll talk about that in the next lecture. The real problem in optimization is to detect small gradients that have an even smaller curvature. Heissan-free Optimization, tailored to your own apps is good at doing that.
The third method really kind of evades the problem. What we do is we carefully initialize the input to hidden weights and we very carefully initialize the hidden to hidden weights, and also feedback weights from the outputs to the hidden units. And the idea of this careful initialization is to make sure that the hidden state has a huge reservoir of weakly coupled oscillators. So if you hit it with an input sequence, it will reverberate for a long time and those reverberations are remembering what happened in the input sequence.
You then try and couple those reverberations to the output you want and so the only thing that learns in an Echo State Network is the connections between the hidden units and the outputs. And if the output units are linear, that's very easy to train. So this hasn't really learned the recurrent. It's used a fixed random recurrent bit, but a carefully chosen one and then just learned the hidden tripod connections.
And the final method is to use momentum, but to use momentum with the kind of initialization that was being used for Echo State Networks and that makes them work even better. So it was very clever to find out how to initialize these recurrent networks so they'll have interesting dynamics, but they work even better if you now modify that dynamic slightly in that direction that will help with the task at hand.
Neural Networks for Machine Learning Lecture 7e Long term short term memory Geoffrey Hinton Nitish Srivastava, Kevin Swersky Tijmen Tieleman Abdel-rahman MohamedLong Short Term Memory (LSTM) Hochreiter & Schmidhuber (1997) solved the problem of getting an RNN to remember things for a long time (like hundreds of time steps). They designed a memory cell using logistic and linear units with multiplicative interactions. Information gets into the cell whenever its “write” gate is on. The information stays in the cell so long as its “keep” gate is on. Information can be read from the cell by turning on its “read” gate. To preserve information for a long time in the activities of an RNN, we use a circuit that implements an analog memory cell.
weight of 1 will maintain its state.
activating its write gate.
the read gate.
circuit because logistics are have nice derivatives. keep gate 1.73 write gate input from rest of RNN read gate output to rest of RNNBackpropagation through a memory cell keep 0 keep 1 keep 1 1.7 1.7 read 0 write 1 1.7 keep 0 1.7 read 0 write 0 time à read 1 write 0 1.7Reading cursive handwriting This is a natural task for an RNN. The input is a sequence of (x,y,p) coordinates of the tip of the pen, where p indicates whether the pen is up or down. The output is a sequence of characters. Graves & Schmidhuber (2009) showed that RNNs with LSTM are currently the best systems for reading cursive writing.
small images as input rather than pen coordinates.A demonstration of online handwriting recognition by an RNN with Long Short Term Memory (from Alex Graves) The movie that follows shows several different things: Row 1: This shows when the characters are recognized.
Row 2: This shows the states of a subset of the memory cells.
Row 3: This shows the writing. The net sees the x and y coordinates.
Row 4: This shows the gradient backpropagated all the way to the x and y inputs from the currently most active character.
In this video, I'm going to talk about the back propagation through time algorithm. It's the standard way to train or recurrence your own network. The algorithm is really quite simple once you have seen the equivalents between a recurrent neural network and a feed forward neural network that has one layer for each time step. I'll also talk about ways of providing input, and desired outputs, to recurrent neural networks. So the diagram shows a simple recurrent net with three interconnected neurons. We're going to assume there's a time delay of one in using each of those connections and that the network runs in discrete time, so the clock that has integer ticks. The key to understanding how to train a recurrent network is to see that a recurrent network is really just the same as a feed forward network, where you've expanded the recurrent network in time. So the recurrent network starts off in some initial state. Shown at the bottom there, times zero. And then uses the way some of these connections to get a new state, shown at time one. You then uses the same weights again to get another new state, and it uses the same weights again to get another new state and so on. So it's really just a lead feed forward network, where the weight is a constraint to be the same at every layer. Now backprop is good at learning when there are weight constraints. We saw this for convolutional nets and just to remind you, we can actually incorporate any linear constraint quite easily in backprop. So we compute the gradients as usual, as if the weights were not constrained. And then we modify the gradients, so that we maintain the constraints. So if we want W1 to equal W2, we start off with an equal and then we need to make sure that the changing W1 is equal to the changing W2. And we do that by simply taking the derivative of the area with respect to W1, the derivative with respect to W2, and adding or averaging them, and then applying the same quantity for updating both W1 and W2. So if the weights started off satisfying the constraints they'll continue to satisfy the constraints. The backpropagation through time algorithm is just the name for what happens when you think of a recurrent net as a lead feet forward net with shared weights, and you train it with backpropagation. So, we can think of that algorithm in the time domain. The forward pass builds up a stack of activities at each time slice. And the backward pass peels activities off that stack and computes error derivatives each time step backwards. That's why it's called back propagation through time. After the backward pass we can add together the derivatives at all the different time step for each particular weight. And then change all the copies of that weight by the same amount which is proportional to the sum or average of all those derivatives. There is an irritating extra issue. If we don't specify the initial state of the all the units, for example, if some of them are hidden or output units, then we have to start them off in some particular state. We could just fix those initial states to have some default value like 0.5, but that might make the system work not quite as well as it would otherwise work if it had some more sensible initial value. So we can actually learn the initial states. We treat them like parameters rather than activities and we learn them the same way as learned the weights. We start off with an initial random guess for the initial states. That is the initial states of all the units that aren't input units And then at the end of each training sequence we back propagate through time all the way back to the initial states. And that gives us the gradient of the error function with respects to the initial state. We then just, adjust the initial states by following, that gradient. We go downhill in the gradient, and that gives us new initial states that are slightly different. There's many ways in which we can provide the input to a recurrent neural net. We could, for example, specify the initial state of all the units. That's the most natural thing to do when we think of a recurrent net, like a feed forward net with constrained weights. We could specify the initial state of just a subset of the units or we can specify the states at every time stamp of the subset of the units and that's probably the most natural way to input sequential data. Similarly, there's many way we can specify targets for a recurrent network. When we think of it as feed forward network with constrained weights, the natural thing to do is to specify the desired final states for all of the units. If we're trying to train it to settle to some attractor, we might want to specify the desired states not just for the final time steps but for several time steps. That will cause it to actually settle down there, rather than passing through some state and going off somewhere else. So by specifying several states of the end, we can force it to learn attractors and it's quite easy as we back propagate to add in derivatives that we get from each time stamp. So the back propegation starts at the top, with the derivatives for the final time stamp. And then as we go back through the line before the top we add in the derivatives for that man, and so on. So it's really very little extra effort to have derivatives at many different layers. Or we could specify the design activity of a subset of units which we might think of as output units. And that's a very natural way to train a recurrent neural network that is meant to be providing a continuous output.


This module implements the binary protocol used by the Network plugin to let you send arbitrary numeric data to collectd servers. Other than turning on the Network plugin on the destination collectd server, no configuration is needed.
Although you can configure your collectd server to do anything with your statistics, most people use the RRDtool plugin to efficiently store large quantities of data. Therefore, this tutorial will discuss the output files generated by the RRDtool plugin, even though this module sends destination-agnostic statistics messages to your collectd server.
Here’s an example of sending statistics:
import time, random
import collectd
collectd.start_threads()
conn = collectd.Connection()
while True:
conn.some_category.record(some_counter = 1, another_stat = random.random())
if random.randrange(2):
conn.coin_stats.record("heads", flips = 1)
else:
conn.coin_stats.record("tails", flips = 1)
time.sleep(random.randint(1, 4))
If you run this script (which is examples/basic.py in your source package) then your collectd server will create an RRD file for each statistic you are tracking. These files are typically stored under one central location on the filesysem, such as /var/lib/collectd/rrd and organized by the hostname of where the statistics came from and the name of the plugin that generated them.
This module identifies itself with the plugin name any, so if you ran the above script, it would create a localhost/any directory with the following files:
Each call to record() increments the statistics you provide. Periodically the sums of these statistics are sent to the collectd server and then reset. Therefore, if you displayed a graph of one of these statistics files, each data point on the graph would represent the sum of all record() values over that time increment.
This module is free for use under the BSD license. It requires Python 2.6 or Python 2.7 and has no other dependencies.
You may click here to download the collectd module. You may also run easy_install collectd if you have EasyInstall on your system. The project page for collectd in the Cheese Shop (aka the Python Package Index or PyPI) may be found here.
You may also check out the development version of collectd with this command:
git clone https://github.com/appliedsec/collectd.git
This function starts two daemon threads. The first takes snapshots of your counters and resets them periodically. The other serializes the statistics into protocol messages understood by the collectd Network plugin and sends them in appropriately-sized UDP packets.
You must call this function when your program starts, or else this module will never actually send any data to any collectd servers. Calling this function more than once throws an exception.
Connection objects may be instantiated with 5 optional arguments:
Connection objects with identical parameters are singletons; in other words, Connection("foo") is Connection("foo") but Connection("foo") is not Connection("bar").
Statistics are recorded through Counter objects, which are dynamically generated by accessing fields on Connection instances. So saying conn.foo creates a Counter object.
These objects are distinct but cached, so conn.foo is conn.foo but conn.foo is not conn.bar.
You shouldn’t directly instantiate this class; instead Counter objects are automatically created by accessing attributes of Connection objects such that conn.foo will cache and return Counter("foo").
Both of the following methods swallow and log all possible exceptions, so you never need to worry about an error being thrown by calls to either of these methods. These functions are also synchronized, so you can safely call them simultaneously from different threads.
Each keyword argument to this method is interpreted as a statistic, whose name is the argument name, and whose value is incrementd by the argument value.
You may pass one or more string identifiers as positional arguments to this method. If you do so, then separate statistic counts will be maintained for this identifier. These counts are always added to the base count for each statistic.
For example, if you ran the following code
conn = collectd.Connection()
conn.example.record(baz = 1)
conn.example.record("foo", baz = 2)
conn.example.record("bar", baz = 3)
and then statistics were sent to the collectd server, it would result in the following files:
Each keyword argument to this method is interpreted as a statistic, whose name is the argument name, and whose value is set to the exact value of the argument. Use this method when you have values which you wish to update to a specific value rather than increment.
Warning
All names are sanitized to contain only alphanumeric characters separated by underscores. In other words, if you did something like
conn = collectd.Connection()
counter = getattr(conn, "^@#@foo%$&&*")
stat = {"*()#@spam!@$^&*": 1}
counter.record("()#@bar&*()baz$^_+", **stat)
then the resulting files would be named gauge-foo-spam.rrd and gauge-foo-bar_baz-spam.rrd. Although this behavior is generally desirable, it could lead to your statistics becoming merged. For example, the foo-bar and foo/bar statistics in the following code would be combined into a single foo_bar statistic:
conn = collectd.Connection()
conn.foo.record("foo-bar", baz = 2)
conn.foo.record("foo/bar", baz = 3)
As mentioned above, collectd swallows exceptions so that you never have to worry about calling a function from this module and triggering an exception. However, any exceptions which do occur are logged using the logging module from the standard library, specifically to the collectd logger. No other logging is performed.
However, this module defines no handlers for the collectd logger, so by default nothing is done with any of these log messages.
The following code is available in the examples/primes.py file in your source distribution:
import time
from Queue import Queue
from threading import Thread
from random import normalvariate
import collectd
numbers = Queue()
conn = collectd.Connection()
def is_prime(n):
for i in xrange(2, n):
if n % i == 0:
return False
return True
def watch_queue():
while True:
conn.queue.set_exact(size = numbers.qsize())
time.sleep(1)
def consumer():
while True:
n = numbers.get()
before = time.time()
primality = is_prime(n)
elapsed = time.time() - before
if primality:
print n, "is prime"
conn.consumer.record("prime", count = 1, time = elapsed)
else:
print n, "is not prime"
conn.consumer.record("composite", count = 1, time = elapsed)
def producer():
while True:
n = int((time.time() % 30) ** normalvariate(5, 2))
if n < 2:
conn.producer.record(too_small = 1)
elif n > 10 ** 9:
conn.producer.record(too_big = 1)
else:
conn.producer.record(just_right = 1)
numbers.put(n)
time.sleep(0.33)
if __name__ == "__main__":
collectd.start_threads()
for func in [producer, consumer]:
t = Thread(target = func)
t.daemon = True
t.start()
watch_queue()
Here’s a list of files generated by this code, along with an explanation of the counter which each file contains for each time interval:
TODO: add graphs to this example to demonstrate what the data looks like
This is forthcoming; we’d like a simple script for generating the complex command-line arguments understood by rrdtool graph commands.
At some point we may also want a web interface similar to the existing collectd cgi script which dynamically lets you select and generate graphs, but with awareness of the semantics of the files generated by this module, which would allow users to start with a high-level overview and drill-down into specific data, combine data from different sources into one graph, etc.


documentation
The documentation for Thymol 2.0 is very much a "work-in-progress".
Please share your comments, corrections, suggestions for improvements and contributions by email to:
docs AT thymoljs.org
Please check again later for more content.
--- title: Thymol Documentation description: Detailed descriptions of configuration and use of Thymol --- {% include header-section.html level="" %}
{% include common-section.html %}The documentation for Thymol 2.0 is very much a "work-in-progress".
Please share your comments, corrections, suggestions for improvements and contributions by email to:
docs AT thymoljs.org
Please check again later for more content.


---
sort_order: 186
tei_id: rdx_b5j2m.p.idp1653168
annotation_count: 0
images:
small-thumbnail: http://readux.library.emory.edu/books/emory:b5j2m/pages/emory:p79q0/mini-thumbnail/
json: http://readux.library.emory.edu/books/emory:b5j2m/pages/emory:p79q0/info/
full: http://readux.library.emory.edu/books/emory:b5j2m/pages/emory:p79q0/fs/
page: http://readux.library.emory.edu/books/emory:b5j2m/pages/emory:p79q0/single-page/
thumbnail: http://readux.library.emory.edu/books/emory:b5j2m/pages/emory:p79q0/thumbnail/
title: Page 186
number: 186
---
{% raw %}182 MRS. II. BEECHER STOWE's " UNCLE luM."{% endraw %}
{% raw %}the river's bank; he saw the boat half a mile off; he{% endraw %}
{% raw %}hailed it, and said, " Is there a man named Smith on{% endraw %}
{% raw %}that boat?" "I'm the man, sir." " Come ashore,.{% endraw %}
{% raw %}I want to speak to you." This Smith then told the{% endraw %}
{% raw %}magistrate that he had worked with Alexander, and{% endraw %}
{% raw %}that " he was a mean, lying thief, and he could{% endraw %}
{% raw %}prove it." "Enough; I subpoena you to appear at{% endraw %}
{% raw %}the court this morning by nine o'clock to testify in{% endraw %}
{% raw %}this case," Squire Terrace answered.{% endraw %}
{% raw %}The time came. It was understood at court that{% endraw %}
{% raw %}a witness would testify to the character of John{% endraw %}
{% raw %}Alexander, who was present in an exultant frame of{% endraw %}
{% raw %}mind. The witness was called. The attorney said,{% endraw %}
{% raw %}" You have worked with John Alexander; is he{% endraw %}
{% raw %}a trustworthy man ? Has he a good, reliable cha¬{% endraw %}
{% raw %}racter?" Smith said, "He is one of the greatest{% endraw %}
{% raw %}rogues out of prison." Alexander was about to{% endraw %}
{% raw %}interrupt him ; but Smith looked him square in the{% endraw %}
{% raw %}face, and said, " You know if you stepped your{% endraw %}
{% raw %}foot where you used to work with me, you'd be{% endraw %}
{% raw %}hustled off to prison, where you ought to go if you{% endraw %}
{% raw %}got your deserts." Squire McDonald exclaimed,{% endraw %}
{% raw %}" What do you say ? Is the man a rogue—has he{% endraw %}
{% raw %}no character?"{% endraw %}
{% raw %}" He has none, sir; but was obliged to ' cut{% endraw %}
{% raw %}sticks,' as we say up in the country—that is, he gave{% endraw %}
{% raw %}' leg bail' and ran away."{% endraw %}
{% raw %}" Well," said Squire McDonald, " if John Alex¬{% endraw %}
{% raw %}ander has no character, Mr. Henson has his{% endraw %}
{% raw %}acquittal." He was as much astounded at the{% endraw %}
{% raw %}appearance of the witness as my accuser was. It{% endraw %}
{% raw %}is needless to say that my friends and family re-{% endraw %}


driver sht31 interface header file
uint8_t sht31_interface_iic_init(void)
interface iic bus init
uint8_t sht31_interface_iic_read_address16_with_scl(uint8_t addr, uint16_t reg, uint8_t *buf, uint16_t len)
interface iic bus read with 16 bits register address
uint16_t sht31_interface_debug_print(char *fmt,...)
interface print format data
uint8_t sht31_interface_iic_read_address16(uint8_t addr, uint16_t reg, uint8_t *buf, uint16_t len)
interface iic bus read with 16 bits register address
uint8_t sht31_interface_iic_deinit(void)
interface iic bus deinit
uint8_t sht31_interface_iic_write_address16(uint8_t addr, uint16_t reg, uint8_t *buf, uint16_t len)
interface iic bus write with 16 bits register address
void sht31_interface_delay_ms(uint32_t ms)
interface delay ms


Aloe Blacc - I Need A Dollar
Welcome to my personal website
Also serving as a digital portfolio
Introduction
Software development & Car Mechanic/Modification
Tid bits:
Ability to also read and speak Portuguese
Moderate ability to understand Spanish
Employment
Medical Records Tech at CioxHealth 2019-2020
Customer Service Rep at CircleK 2021-2023
Currently: Studying full time at Mt. San Jacinto College
Language usage experience:
Python
HTML, CSS, and javascript
AHK, Batch and Lua
Selling:
Programs, Scripts, and bots
Websites
Car Mechanic and Modification services
Other information and certs
Certs and training completed during my employment at CioxHealth:
Medicare FWA training
Medicare parts C and D compliance training
ADA compliance
HIPAA training
Contact Options listed in best way to contact me:
The place you found my offer from
Steam
Discord: momomeomo
masters bachelors PHD computer science degree college math


In the Cambridge project, Lin Shi yuan is the bridge between the government and the people. The remediation process will bring a lot of inconveniences and noises, so Chief Lin not only had to listen to the needs of the people, but also to communicate with the city hall. Chief Lin played an important role in environmental protection. He assisted to organize local people to maintain the cleanness of the waterfront.
We asked Chief Lin a question, "Why can the water quality in Cambridge still not get improved?" Chief Lin answered, " With regard to the sewage treatment section, at present, domestic wastewater is discharged directly into the river while industrial wastewater is treated first. "
Chief Lin added, “the Cambridge Rezoning Plan will be expanded to Big Cambridge Project. That is, the city hall and the central government have passed a budget to treat domestic wastewater on a bigger scale. Yet, it will not be completed until next year. This project also aims to make Taichung become a base to promote canoeing sport and strive to make the canoeing as an official competition in 2019 East Asian Youth Games.”
Talking about the volunteer team organization, Chief Lin said: "The volunteer members are 60 in total. There are one captain, two vice-captains and one financial officer. They are divided into four teams of 15 people and one team captain. Although they are local volunteer teams, they systematically gear up to help make Cambridge a better place.”
6/24, 8/27, 2017 Water Sport Carnival
8/24-26,2017
8/24 Thur.【 Shinning Starry Sky 】 National Taiwan Symphony Orchestra with orchestra
8/25 Fri. 【 Blowing Wind 】 Extreme physical dance group
8/26 Sat. 【 Mozart's Adventure 】 NTSO &American magic ring mime theater group
12/31, 2017 Saxophone Concert
We attended the activity on 8/26. Having such a nice place to watch an international show is real a blessing. We found that all the citizens all appreciated the new waterfront and had a romantic time as they did in western countries. We really owe what we enjoy to all the people who construct and maintain Cambridge.


The barred owl pictured here was just feet from the main path in Explore the Wild. The bird was being harassed by crows, but upon my arrival the corvids retreated leaving the owl in quiet solitude. The owl remained on its perch most of the day looking, as owls often do, peaceful, content, and wise. Read more
Quiet Winter
It’s February, and so far this winter we’ve skated by with very few cold days—no ice storms, only one brief snow, and minimal frigid NW winds. That could change at any time, but for now, I’m going to enjoy the mild fifty, sixty, and yes, even seventy degree weather. The bird feeders have seen steady, but not heavy, use. The local chickadees, titmice, cardinals, nuthatches, and others have been regular customers at the feeders at Bird Viewing, but I’ve not seen any Read more
Winter News
The temperatures have taken a nose dive. It’s a good twenty degrees colder today than it was yesterday—and will remain so for the next week or more. The high today will be near 50º which is average for this time of year. If you lived up north, close to the Canadian border, that’d be a shirtsleeve day for sure. But we’re here in North Carolina, and after a week or more of 60’s and 70’s (with lows around fifty), it Read more
The Pack That Howls Together…
If you’re lucky enough to be in Explore the Wild when our wolves begin to howl, you’re in for a treat. You’re heart will quicken. An element of wonderment, perhaps a tinge of fear, will stir in you. Adrenaline will pump through your veins, fight or flight. It’s a wild sound, an emotional sound. It’s a sound of the wilderness. When wolves howl, they’re communicating with one another. That communication may be an attempt to locate or socialize with members of their Read more
A Familiar Face and a New Face
What is becoming an increasingly familiar face here at the Museum is that of the local barred owl. I’ve encountered this owl seven times since October, each time alerted to its presence through the harassing calls of crows, blue jays and various dickey birds. This time, the owl was in the swamp across from the Main Wetlands Overlook in Explore the Wild. The owl seemed to be resting, but apparently at least one eye and both ears were acutely tuned Read more
Quick Quiz
Take a look at the photo below and see if you can identify the subject. If you said “the tail of an eastern gray squirrel,” you’d be correct.  Eastern gray squirrels (Sciurus carolinensis) are everywhere in the east. From east Texas to Saskatchewan and east to the Atlantic Coast gray squirrels are a familiar sight. Read more
What Happened Here?
Near the Wetlands, and next to an American holly loaded with berries, stands a sapling elm tree. There are many such trees here at the Museum. But, as I walked past this particular pair of arboreal specimens I noticed several clusters of passerine contour feathers stuck to the thin branches of the small, bare elm. Most of the feathers were white, some had rufus colored centers. What happened here? When I see a group of feathers clumped together as on the Read more
Red Wolf Forecast
If you’ve been following events here at the Museum you no doubt already know that we have a new female red wolf. She came to us from Reflection Riding Arboretum & Nature Center in Chattanooga, TN back in November with the hopes of breeding with our male red wolf. Whether or not these two wolves come together is entirely up to the individuals involved, the wolves themselves. With cautious optimism, I will say that they seem to be getting along Read more
What’s Going On Outside
If you’re walking the paths on a regular basis here at the Museum, you’re likely to see all manner of creature, winter, spring, summer, or fall. All of the creatures pictured below were photographed within the last few weeks. Brown snakes are common in this area. They attain lengths of approximately 12 inches, although the record is just over 19 inches. The individual in the photos above and below is a young one and is 6 to 7 inches. It was Read more
A Glimpse of the Past
I was walking down the path that leads from Catch the Wind to Explore the Wild on the far side of that half mile exhibit loop. I glanced to my right and noticed a small tan-colored object dangling from a twig on a hornbeam, or ironwood, tree on the right side of the path, chrysalis. The tree is very close to a patch of partridge pea, which is host to cloudless sulphur butterflies in late summer and fall. I saw Read more
Bufflehead
On December 15, I noticed a small gray duck swimming in proximity to the slightly larger hooded mergansers in our Wetlands. I immediately thought bufflehead. The last time I saw a bufflehead in the Wetlands was November of 2014 when two showed up. At least one of them stayed until 26, December. Sure enough, there was another this year. With the current freezing temps, all waterfowl had departed by the end of this week. Diving ducks, of which both mergansers and Read more
Handful of Herons
I’m sometimes asked how many species of bird I’ve seen here at the Museum. Currently, the list is over 130 species. Of those, five species have been herons, great blue heron, great egret, green heron, and black-crowned and yellow-crowned night-herons. A great blue heron, or two, can be seen in our wetlands year-round. Although some great blue herons migrate, they can be found in our area in any season. Great egrets are widespread in North Carolina but are essentially a Read more


ChiantiPy
is a Python interface to the
CHIANTI atomic database for
astrophysical spectroscopy.
CHIANTI consists of a database of atomic data that can be
used to interpret spectral lines and continua emitted from
high-temperature, optically-thin astrophysical sources.
Mailing list:
users


MATT RYAN
02.13.17
Ohio University was one of the top five party schools in the country according to The Princeton Review when OU President Roderick McDavis started his time in Cutler Hall, and OU became the top party school during the middle of his presidency.
Since securing the top spot, OU has been slipping in the party school rankings, and what once was the top party school in the country did not even crack the The Princeton Review’s most recent party school rankings. OU is not absent from all party school rankings, however, as Playboy named OU the top party school for 2015.
“A hallmark of President McDavis’ legacy has been altering the high-risk drinking culture at Ohio University.” Dan Pittman, OU Spokesman
Athens Police Chief Tom Pyle said he has noticed a difference in the party culture during McDavis’ time as OU’s president.
"Certainly the fests reflect in my mind a significant change in party culture,” Pyle said. “Noise complaints are down so I think our enforcement in that regard has had an impact."
Although arrests during fests have also been decreasing in recent years, they have fluctuated up and down throughout McDavis’ presidency. Nine students were arrested during 2005’s Palmer Fest, 74 people were arrested during Palmer Fest in 2009 and 14 people were arrested during 2016’s Palmer Fest, which does not include arrests from the Ohio Investigative Unit.
Matt Pruetz, a senior studying management information systems, said fest season helps make OU a party school.
“Not a lot of other schools do that,” Pruetz said. “If they do have (fests), it's usually one time, not a whole season."
Recent fests haven’t seen the fires that had previously been a reoccurring event at Palmer Fest during McDavis’ presidency. The fest ended in flames in 2012 because of a basement fire at one of the houses, and it was later declared a riot by then-Athens Mayor Paul Wiehl. The fire, which was later deemed to be arson, broke out at about 7 p.m. In 2010, partygoers set a couch on fire after 11 p.m. during Palmer Fest.
In addition to fewer arrests, parties during fest season have been shutting down earlier in recent years as well. During 2016’s fest season, most parties during Palmer Fest were shut down by 5:30 p.m.
The Ohio University Police Department has also noticed a difference during McDavis’ presidency, OUPD Lt. Tim Ryan said.
“I think that we have seen less of a party atmosphere,” Ryan said.
Arrests for Halloween have also been down lately. During McDavis’ first Halloween in 2004, police arrested 98 people. Arrests rose for 2005’s Halloween to nearly 150. In 2010, there were 181 arrests, and in 2011, there were 148 arrests, which officials described as "quiet" compared to previous years. APD arrested 35 people during the 2016 Halloween block party, OUPD issued 69 citations and the Ohio Department of Public Safety Investigation Unit charged approximately 50 people.
Pyle said there were too many factors in play to pinpoint one aspect that has changed the party culture at OU.
“It was a concentrated effort to try to impact the culture of partying and drinking,” Pyle said.
Pruetz said he has noticed a difference in the party culture during his time at OU.
"When I was younger ... it was more house party based a lot,” Pruetz said. “I remember going to parties during the weekdays. Now it's definitely bar central.”
Anna McCall, a freshman studying special education, said she thinks OU’s party school reputation is “way overrated.”
“I've been to other colleges around and it's no different there," McCall said.
She said she thinks university officials are trying to change OU’s reputation as a party school.
“It seems like they'll be cracking down here on that and just trying to get rid of the party school reputation," McCall said.
All incoming OU students are required to complete AlcoholEdu, an online alcohol education course. Since 2005, It has been mandated for first-year students and transfer students with fewer than 30 credit hours, Pittman said.
AlcoholEdu cost the Division of Student Affairs $45,000 for the 2015-16 academic year, Pittman said.
Nearly 55 percent of OU students identified as moderate drinkers, about 16 percent identified as non-drinkers and nearly 13 percent identified as abstainers, according to the Fall 2016 Healthy Campus Survey. The survey, which started fall 2015, is a tool used by the university to monitor student alcohol consumption.
A moderate drinker is categorized as one to four drinks for males and one to three drinks for females, a nondrinker is zero drinks in the past two weeks and an abstainer has not had a drink in the past year.
About 10 percent of students identified as heavy episodic drinkers, and about 7 percent identified as problematic drinkers, according to the Fall 2016 Healthy Campus Survey.
A heavy episodic drinker is categorized as consuming five to nine drinks for males and four to seven drinks for females, and a problematic drinker is categorized as consuming 10 or more drinks for males and eight or more drinks for females.
Dusty Moell, a freshman studying business, said OU’s reputation as a party school is overhyped.
"You hear OU, people are automatically like 'oh, it's a party school,' but you can really find parties anywhere," Moell said.
He said he first learned about OU’s party school reputation during his junior year of high school after hearing about Halloween.
"I think it helps that other schools have picked up the slack,” Moell said. “Maybe it hasn't decreased as much here as it has increased other places as well.”
Landing Page
OU President Roderick McDavis is leaving February 17. The Post looks back at his time as Ohio's president. Click this box to read the rest of the stories from this issue.


Here is a write-up for how I match the MSCC sequencing data and generate the number of counts.
Rather than blasting against the genome, I generate a list of all sequences I expect I could see. I can do this because I know all tags should be generated from a CCGG sequence and because most of the genome has been sequenced. (The parts that aren't are highly repetitive and presumably would generate tags matching known sequences considered to be non-unique.)
To do this, start with a fasta-formated genome sequence data file. Each sequence header should start with a chromosome ID that contains no whitespace (eg. "chr1_random"). The following perl script can be used to generate a list of all target locations with flanking sequence and their locations:
The resulting data will look something like this:
chr1 3004692 CCACACGATCTTAGGACCTCCAGTGGAACACAACTTCTGTTCCAATCTAATCGTGCAAGACCGGAGACAGCATTAAGGAAGCAGAAAACCAGCCTGACCAGGGTCACAAGTCCCTTCTGGATGG 1 chr1 3005055 ATTTGCAGTGATGGCCGACTAGGCCATCTTTTGATACATATGCAGCTAGAGACAAGAGCTCCGGGGTACTAGTTAGTTCATATTGTTATTCCACCTATAGGGTTGCAGTTCCCATTAGCTCCTT 1 chr1 3005766 CAGCTTCTGGCTATTATAAACAAGGCTGCTATGAACACAGTGGAGCATGTGTCCTTCTTACCGGTTGGGACATCTTCTGGATATATGCAAAGGAGAGGTATTGTGGGATCCTTAGGTTACTATG 1 chr1 3006702 TGTCACTATACCAGTACCATGCAGTTTTGATCACAATTGCTCTGTAGTACAGTTTTAGGTCCGGCATAGTGATTCCACCAGAGGTTCTTTTATCCTTGAGAAGAGTTTTTGCTATCCTCGTTTT 1 chr1 3010875 CAGAAACTGCTGGCCTCTGTATTCCACACCCTCACCCATGCAGCCTGCCCTCCTCAGAGTCCGGAACCAAGGTGGCTCCTGCGGAGCCTGAGGCAGAAACCTCTTGGGCCGGGTGGACCCCTGT 1 chr1 3010923 CCTCCTCAGAGTCCGGAACCAAGGTGGCTCCTGCGGAGCCTGAGGCAGAAACCTCTTGGGCCGGGTGGACCCCTGTGCTCTCACCAGGAAGGTGGCCGGTTGTCTGTAGCCGAAAATGGCACCA 1 chr1 3010958 GAGCCTGAGGCAGAAACCTCTTGGGCCGGGTGGACCCCTGTGCTCTCACCAGGAAGGTGGCCGGTTGTCTGTAGCCGAAAATGGCACCACCTCAGAAGCTCTGTGGCTCTTGCCTGTCCCAGAA 1 chr1 3011026 CTGTAGCCGAAAATGGCACCACCTCAGAAGCTCTGTGGCTCTTGCCTGTCCCAGAAACCCCCGGCCCCTGCACTCCACACCCTCACCCGTGCAGCCTGCCCTCCGTGGAGTCCTGGAGCCAAGG 1 chr1 3011093 CTGCACTCCACACCCTCACCCGTGCAGCCTGCCCTCCGTGGAGTCCTGGAGCCAAGGTGGCCGGTTGTCTCCCACGTGATCTTAAAACCACTAACCAGACAGCTCCATGCTCCTCAAAGAAGAC 1 chr1 3016501 TGCCACCAAGATTTTGCTGAAAGGACACTGATATTGCTGTCTCTTGTGATGCTACGTATGCCGGTGCCTGGGAAACACAGAAAAATGGATGCTCACAGTCAGCTATTGGATGGAACACAGGGCC 1
The last column marks whether the target site was in repetitive DNA by checking if it was "repeat-masked" in the genome sequence data file (indicated by being lowercase). 1 if in repeat-masked (lowercase) sequence, 0 if not.
Now we want to generate a list of all possible tags that could be generated. There are two major issues here:
In our library, the design placed the sequencing primer opposite to the target cut site, the expected tag sequences end (rather than begin) at the target site.
Because Mme I has a wobble when it cuts (it cuts in 20 or 21 bp, about 50% time for each), two lists will be generated. In our Hpa II MSCC design these were 18 and 19 basepair variants. (This is 18/19 rather than 20/21 because two of the bases are always the same -- the "GG" in the CCGG site.)
To do this we use the file generated from the previous step and the following script:
The resulting data will look something like this:
CAATCTAATCGTGCAAGA chr1 3004692 - 18 1 NA GCTTCCTTAATGCTGTCT chr1 3004692 + 18 1 NA CAGCTAGAGACAAGAGCT chr1 3005055 - 18 1 NA TATGAACTAACTAGTACC chr1 3005055 + 18 1 NA GAGCATGTGTCCTTCTTA chr1 3005766 - 18 1 NA ATCCAGAAGATGTCCCAA chr1 3005766 + 18 1 NA TGTAGTACAGTTTTAGGT chr1 3006702 - 18 1 NA TCTGGTGGAATCACTATG chr1 3006702 + 18 1 NA GCCTGCCCTCCTCAGAGT chr1 3010875 - 18 1 NA GCAGGAGCCACCTTGGTT chr1 3010875 + 18 1 NA
This shows the tag sequence, the chromosome ID, position, strand (upstream or downstream tag), tag length, if it's repeat-masked, and MmeI site presence. The last column, MmeI site presence, is "NA" if no conflicting MmeI site is found in the flanking sequence or, if one is detected, the distance in bp to the target site. This is recorded here because if there is an MmeI site in genomic sequence it could conflict with the tag construction and so sites with MmeI sites that are too close should be excluded in later analysis (although they are potentially generated and so should be in our set of potential matches).
Because we'll want to exclude analysis of sites too close to each other (and so they would interfere with each other) we want to add a column to the data indicating distance to the neighboring site. We did this with command line perl code like the following (note that this code assumes that the upstream sites precede the downstream sites in the input file):
cat CCGG_tags_18 | perl -ne 'BEGIN {$dist = 1000;} @data = split; if ($data[3] eq "-") {if ($data[1] ne $olddata[1]) { if (@olddata) { print<EMAIL_ADDRESS>1000 $data[6]\n";}; print<EMAIL_ADDRESS>1000 $data[6]\n";} else {$dist = $data[2] - $olddata[2]; print<EMAIL_ADDRESS>$dist<EMAIL_ADDRESS>$dist $data[6]\n";} }; @olddata = @data; END {print<EMAIL_ADDRESS>1000 $data[6]\n";}' > CCGG_tags_18_withdist
This ends up looking like this:
CAATCTAATCGTGCAAGA chr1 3004692 - 18 1 1000 NA GCTTCCTTAATGCTGTCT chr1 3004692 + 18 1 363 NA CAGCTAGAGACAAGAGCT chr1 3005055 - 18 1 363 NA TATGAACTAACTAGTACC chr1 3005055 + 18 1 711 NA GAGCATGTGTCCTTCTTA chr1 3005766 - 18 1 711 NA ATCCAGAAGATGTCCCAA chr1 3005766 + 18 1 936 NA TGTAGTACAGTTTTAGGT chr1 3006702 - 18 1 936 NA TCTGGTGGAATCACTATG chr1 3006702 + 18 1 4173 NA GCCTGCCCTCCTCAGAGT chr1 3010875 - 18 1 4173 NA GCAGGAGCCACCTTGGTT chr1 3010875 + 18 1 48 NA
Since our later matching requires this file be alphabetically sorted, you can do this with a unix command:
sort CCGG_tags_18_withdist > CCGG_tags_18_withdist_alphasort
Which should get you something that looks like this:
AAAAAAAAAAAAAAAAAA chr10 62355617 + 18 0 1363 NA AAAAAAAAAAAAAAAAAA chr10 75423711 + 18 1 1459 NA AAAAAAAAAAAAAAAAAA chr11 107327557 - 18 1 593 NA AAAAAAAAAAAAAAAAAA chr11 118896086 - 18 0 73 NA AAAAAAAAAAAAAAAAAA chr11 39946525 + 18 0 168 NA AAAAAAAAAAAAAAAAAA chr12 110522275 - 18 1 8680 NA AAAAAAAAAAAAAAAAAA chr12 56223772 - 18 1 765 NA AAAAAAAAAAAAAAAAAA chr12 56298836 - 18 1 765 NA AAAAAAAAAAAAAAAAAA chr13 21261989 + 18 0 1196 NA AAAAAAAAAAAAAAAAAA chr13 24319036 - 18 0 37 NA
In our final analysis we'll want to only use tags that meet the following criteria:
They are unique enough. We consider this to be true if there are no other exact matches, no other tags with only one base difference, and no more than 10 tags with two bases different.
They are far enough apart. We consider this to be true if the neighboring site more than 50bp distant.
There are no conflicting MmeI sequences that could cut into our library molecule as we try to generate it. We consider this to be true if MmeI sites are more than 50bp distant.
The data needed to evaluate the last two criteria are already recorded within the steps we've taken to generate the list of tags. The first requires matching the tag set against itself. This uses basically the same routine as we use later to match sequencing reads against the tags. Rather than use BLAST, we used a combination of unix commands and perl scripts to find all matches to the target list up to N bases distant for a given sequence. Because any tags that are unique with only 18bp of sequence should also be unique with 19bp, we only performed this for the shorter (18bp) tag variants.
Because the later steps (eg. sorting) will break if the initial set is too large, we work with a small set at a time. The lines of the file are shuffled first to avoid subsets being highly enriched for repetitive sequences (because these have more matches they are more intensive). (This doesn't need to be done when matching sequencing reads.) If the system you're on doesn't have "shuf" or "sort --random-sort" commands, this can be done with this combination of command line perl and unix commands:
cat ../tags_from_sites/CCGG_tags_18_withdist | perl -ne '$rand=rand(); print "$rand $_";' | sort -n | perl -ne '@data=split; print "@data[1..$#data]\n";' > CCGG_tags_18_withdist_shuffled
This is done with Unix "head" and "tail" to pull out a piece of the target sequence at a time, eg:
head -32000 reads_all | tail -2000 > reads_subset
Once a subset is made, it is matched against the target library with the following steps:
For each sequence in subset a set of all variants up to N mismatches is generated. For our purposes here we consider up to 2 mismatch variants, which we generated with the perl program:
The output of this adds information regarding the number of mismatches added and what the base changes were, it should look like this:
CCCGAGTTTTTTCTCGTC chr7 149406279 - 18 1 904 NA EXACT 0 ACCGAGTTTTTTCTCGTC chr7 149406279 - 18 1 904 NA 0_C_A 1 GCCGAGTTTTTTCTCGTC chr7 149406279 - 18 1 904 NA 0_C_G 1 TCCGAGTTTTTTCTCGTC chr7 149406279 - 18 1 904 NA 0_C_T 1 AACGAGTTTTTTCTCGTC chr7 149406279 - 18 1 904 NA 0_C_A,1_C_A 2 AGCGAGTTTTTTCTCGTC chr7 149406279 - 18 1 904 NA 0_C_A,1_C_G 2 ATCGAGTTTTTTCTCGTC chr7 149406279 - 18 1 904 NA 0_C_A,1_C_T 2 GACGAGTTTTTTCTCGTC chr7 149406279 - 18 1 904 NA 0_C_G,1_C_A 2 GGCGAGTTTTTTCTCGTC chr7 149406279 - 18 1 904 NA 0_C_G,1_C_G 2 GTCGAGTTTTTTCTCGTC chr7 149406279 - 18 1 904 NA 0_C_G,1_C_T 2
Sort this list alphabetically using unix sort:
sort subset_variants > subset_variants_alphasort
Which should look like:
AAAAAAAAAAAAAAAAAA chr5 8205497 - 18 1 564 NA 1_G_A,17_G_A 2 AAAAAAAAAAAAAAAAAC chr5 8205497 - 18 1 564 NA 1_G_A,17_G_C 2 AAAAAAAAAAAAAAAAAG chr5 8205497 - 18 1 564 NA 1_G_A 1 AAAAAAAAAAAAAAAAAT chr5 8205497 - 18 1 564 NA 1_G_A,17_G_T 2 AAAAAAAAAAAAAAAACG chr5 8205497 - 18 1 564 NA 1_G_A,16_A_C 2 AAAAAAAAAAAAAAAAGG chr5 8205497 - 18 1 564 NA 1_G_A,16_A_G 2 AAAAAAAAAAAAAAAATG chr5 8205497 - 18 1 564 NA 1_G_A,16_A_T 2 AAAAAAAAAAAAAAACAG chr5 8205497 - 18 1 564 NA 1_G_A,15_A_C 2 AAAAAAAAAAAAAAAGAG chr5 8205497 - 18 1 564 NA 1_G_A,15_A_G 2 AAAAAAAAAAAAAAATAG chr5 8205497 - 18 1 564 NA 1_G_A,15_A_T 2
Use unix "join" to find if any of the target sequences in the alphabetically sorted file match these, outputting to a new file. Because a later perl script expect columns to be in a certain order, we set the first file to the the target sequence data and the second to be subset variant data.
join CCGG_tags_18_withdist_alphasort subset_variants_alphasort > subset_variants_matches
The output of this should look like this:
AAAAAAAAAAAAAAAAAA chr10 62355617 + 18 0 1363 NA chr5 8205497 - 18 1 564 NA 1_G_A,17_G_A 2 AAAAAAAAAAAAAAAAAA chr10 75423711 + 18 1 1459 NA chr5 8205497 - 18 1 564 NA 1_G_A,17_G_A 2 AAAAAAAAAAAAAAAAAA chr11 107327557 - 18 1 593 NA chr5 8205497 - 18 1 564 NA 1_G_A,17_G_A 2 AAAAAAAAAAAAAAAAAA chr11 118896086 - 18 0 73 NA chr5 8205497 - 18 1 564 NA 1_G_A,17_G_A 2 AAAAAAAAAAAAAAAAAA chr11 39946525 + 18 0 168 NA chr5 8205497 - 18 1 564 NA 1_G_A,17_G_A 2 AAAAAAAAAAAAAAAAAA chr12 110522275 - 18 1 8680 NA chr5 8205497 - 18 1 564 NA 1_G_A,17_G_A 2 AAAAAAAAAAAAAAAAAA chr12 56223772 - 18 1 765 NA chr5 8205497 - 18 1 564 NA 1_G_A,17_G_A 2 AAAAAAAAAAAAAAAAAA chr12 56298836 - 18 1 765 NA chr5 8205497 - 18 1 564 NA 1_G_A,17_G_A 2 AAAAAAAAAAAAAAAAAA chr13 21261989 + 18 0 1196 NA chr5 8205497 - 18 1 564 NA 1_G_A,17_G_A 2 AAAAAAAAAAAAAAAAAA chr13 24319036 - 18 0 37 NA chr5 8205497 - 18 1 564 NA 1_G_A,17_G_A 2
Now sort the matches by location and strand (chromosome, then position, then strand, then number of matches) so we can later count how many matches each potential tag has:
sort --key=9,9 --key=10n,10 --key=11,11 --key=17n,17 subset_variants_matches > subset_variants_matches_sorted
The output should look like this:
CAGCTAGAGTCAAGAGCT chr10 100119759 + 18 1 1432 50 chr1 6616737 - 18 1 1729 NA EXACT 0 CAGCTAGAGTCAAGAGCT chr10 100138221 - 18 1 796 NA chr1 6616737 - 18 1 1729 NA EXACT 0 CAGCTAGAGTCAAGAGCT chr10 100312287 - 18 1 7020 NA chr1 6616737 - 18 1 1729 NA EXACT 0 CAGCTAGAGTCAAGAGCT chr10 100360124 - 18 1 3976 NA chr1 6616737 - 18 1 1729 NA EXACT 0 CAGCTAGAGTCAAGAGCT chr10 100396169 - 18 1 3381 NA chr1 6616737 - 18 1 1729 NA EXACT 0 CAGCTAGAGTCAAGAGCT chr10 100434568 + 18 1 5611 NA chr1 6616737 - 18 1 1729 NA EXACT 0 CAGCTAGAGTCAAGAGCT chr10 100569577 - 18 1 366 NA chr1 6616737 - 18 1 1729 NA EXACT 0 CAGCTAGAGTCAAGAGCT chr10 100612095 - 18 1 11844 NA chr1 6616737 - 18 1 1729 NA EXACT 0 CAGCTAGAGTCAAGAGCT chr10 100703979 + 18 1 7069 NA chr1 6616737 - 18 1 1729 NA EXACT 0 CAGCTAGAGTCAAGAGCT chr10 100711048 - 18 1 7069 NA chr1 6616737 - 18 1 1729 NA EXACT 0
Now we used this perl program to count how many exact, 1 mismatch and 2 mismatch matches were found:
The output should look like this:
CAGCTAGAGTCAAGAGCT chr1 6616737 - 18 1 1729 NA 23555 7998 4368 CTGGTGAGCCTTCCAATT chr1 16616107 - 18 1 3652 NA 1451 63 6 TGCATTAACTCCGGAACT chr1 17767955 - 18 1 8 NA 1 0 0 GTCTCTTGTGAGACTATG chr1 21863644 + 18 1 4974 NA 21479 10797 3895 AAGCTGCCTATGTAACGG chr1 24768593 + 18 0 3470 NA 1 0 0 TAGTATGTACACTATCAG chr1 25098866 + 18 0 491 NA 1 0 0 CTCTGCTGGCAAGGTAGC chr1 30368211 - 18 1 156 NA 5485 2861 743 GCACACCTACAGTCTCAG chr1 34050544 + 18 0 276 NA 1 0 0 GAGCATGTGTTCTTCTTA chr1 35100530 - 18 1 561 NA 1210 6089 2081 GAGGTATTGTGGGATCCT chr1 35221537 - 18 1 1145 NA 730 2964 16432
In our work we have used a pair of shell scripts to submit these as a set of jobs to the orchestra computing cluster, MetaMatchFinder.pl submitted a job for each subset, and each job ran MatchFinder.pl which performed all the above tasks to find the matches for that subset.
Once all the matching has been done, files can be combined with "cat" and then sorted according to chromosome, position, and strand. For example:
cat CCGG_tags_18_upto2MM_all_counts/CCGG_tags_18_upto2MM_*_*_counts | sort --key=2,2 --key=3n,3 --key=4,4 > CCGG_tags_18_withECstandard_withdist_upto2MM_combined
Finally giving a file for each Mme I cut distance that should have the same number of lines as the starting files. The head for example may look like this:
CAATCTAATCGTGCAAGA chr1 3004692 - 18 1 1000 NA 1 0 0 GCTTCCTTAATGCTGTCT chr1 3004692 + 18 1 363 NA 1 2 2 CAGCTAGAGACAAGAGCT chr1 3005055 - 18 1 363 NA 4162 27404 5476 TATGAACTAACTAGTACC chr1 3005055 + 18 1 711 NA 208 23521 8296 GAGCATGTGTCCTTCTTA chr1 3005766 - 18 1 711 NA 5765 3270 811 ATCCAGAAGATGTCCCAA chr1 3005766 + 18 1 936 NA 4718 3597 1116 TGTAGTACAGTTTTAGGT chr1 3006702 - 18 1 936 NA 723 300 391 TCTGGTGGAATCACTATG chr1 3006702 + 18 1 4173 NA 11 1113 240 GCCTGCCCTCCTCAGAGT chr1 3010875 - 18 1 4173 NA 1 2 9 GCAGGAGCCACCTTGGTT chr1 3010875 + 18 1 48 NA 14 168 405
Now we can use command line perl like the following to pick out which tag locations are "unique":
cat CCGG_tags_18_withECstandard_withdist_upto2MM_combined | perl -ne '@data=split; if ($data[6] > 50 and ($data[7] eq "NA" or $data[7] > 50) and $data[8] == 1 and $data[9] == 0 and $data[10] <= 10) { print "@data\n"; }' > CCGG_tags_unique
This should produce something like this:
CAATCTAATCGTGCAAGA chr1 3004692 - 18 1 1000 NA 1 0 0 TGCCTGTCCCAGAAACCC chr1 3011026 - 18 1 68 NA 1 0 3 GGGTGTGGAGTGCAGGGG chr1 3011026 + 18 1 67 NA 1 0 0 GTCCTGGAGCCAAGGTGG chr1 3011093 - 18 1 67 NA 1 0 2 AGATCACGTGGGAGACAA chr1 3011093 + 18 1 5408 NA 1 0 2 CTTGTGATGCTACGTATG chr1 3016501 - 18 1 5408 NA 1 0 0 AAGGCTGTAACCATAGGA chr1 3017735 - 18 0 1234 NA 1 0 0 TTTGAGACACTGAGTTTC chr1 3017735 + 18 0 1390 NA 1 0 0 TCAGGTGTGGGAGGGCTG chr1 3019125 - 18 1 1390 NA 1 0 6 AATTTTCTAGACACAATC chr1 3024108 + 18 1 1806 NA 1 0 0


In Java, string is basically an object that represents sequence of char values. An array of characters works same as Java string. For example:
is same as:
Java String class provides a lot of methods to perform operations on strings such as compare(), concat(), equals(), split(), length(), replace(), compareTo(), intern(), substring() etc.
The java.lang.String class implements Serializable, Comparable and CharSequenceinterfaces.
CharSequence Interface
The CharSequence interface is used to represent the sequence of characters. String, StringBuffer and StringBuilder classes implement it. It means, we can create strings in java by using these three classes.
The Java String is immutable which means it cannot be changed. Whenever we change any string, a new instance is created. For mutable strings, you can use StringBuffer and StringBuilder classes.
We will discuss immutable string later. Let's first understand what is String in Java and how to create the String object.
What is String in java
Generally, String is a sequence of characters. But in Java, string is an object that represents a sequence of characters. The java.lang.String class is used to create a string object.
How to create a string object?
There are two ways to create String object:
By string literal
By new keyword
1) String Literal
Java String literal is created by using double quotes. For Example:
Each time you create a string literal, the JVM checks the "string constant pool" first. If the string already exists in the pool, a reference to the pooled instance is returned. If the string doesn't exist in the pool, a new string instance is created and placed in the pool. For example:
In the above example, only one object will be created. Firstly, JVM will not find any string object with the value "Welcome" in string constant pool, that is why it will create a new object. After that it will find the string with the value "Welcome" in the pool, it will not create a new object but will return the reference to the same instance.
Note: String objects are stored in a special memory area known as the "string constant pool".
Why Java uses the concept of String literal?
To make Java more memory efficient (because no new objects are created if it exists already in the string constant pool).
2) By new keyword
In such case, JVM will create a new string object in normal (non-pool) heap memory, and the literal "Welcome" will be placed in the string constant pool. The variable s will refer to the object in a heap (non-pool).


--- layout: post title: 'PICNIC07: Julian Bleecker' date: 2007-09-28 00:00:00 categories: interfaces-&-interaction mobile play ---
Went back to the keyboard, the early human/computer interface. Designed to spell the word "typewriter" with the top row of keys, as a tool for salespeople :) Shows guidance computer from the Apollo landing module, which had 18 keys. Contrast this with the space shuttle ("which doesn't even go to the moon"), a massively complicated bank of controls: it's the same thing but with many more buttons. There hasn't been an evolution of the interfaces (shows compulsory iPhone screenshot).
So where did this interface come from? Doug Engelbart; shows patent diagrams for mouse/computer diagram.
How would computers see us? A being with one eyeball, a huge finger, etc. How might viewing how the computer sees us change the interface in more playful ways. Looks at traditional playground play: physical activity becoming a game controller. Shows children playing a meteors-like game in real time.
He's not a gamer, but is more interested in their playful characteristics. To play WoW, he'd want to be able to take a hike or go for a bike ride and have that be of value within the game experience. Talks about Nike+.


Social Distancing:
Incubation Period:
Contagious Period:
Run Sim
Number of Points:
200
Healthy:
200
Contagious:
0
Hospitalized:
0
Dead:
0
Recovered:
0


ScreenShape extends ScreenMesh
2D primitive. Screen shape can create 2d shapes (Currently only rectangles and circles).
NOTE: ScreenShape instances are not automatically garbage collected.
Static Properties
ScreenShape.SHAPE_RECT = 1
Integer
No description.
ScreenShape.SHAPE_CIRCLE = 2
Integer
No description.
ScreenShape.SHAPE_CUSTOM = 4
Integer
No description.
Properties
strokeEnabled
Boolean
If set to true, the shape will be drawn over with a stroke.
strokeColor
Color
Color of the shape stroke.
strokeWidth
Number
Width of the shape stroke.
Functions
ScreenShape ( Integer shapeType , Number option1 , Number option2 , Number option3 , Number option4 )
nil
Create a new shape of specified type and size/options.
Parameters
shapeType
Integer
Type of shape to create. Currently the only options are ScreenShape.SHAPE_RECT and ScreenShape.SHAPE_CIRCLE.
option1
Number
Width option.
option2
Number
Height option.
option3
Number
Number of vertices for the the circle (defaults to 360). Unused for rectangle.
option4
Number
Reserved.
Clone ( Boolean deepClone , Boolean ignoreEditorOnly )
Entity
No description.
Parameters
deepClone
Boolean
No description.
ignoreEditorOnly
Boolean
No description.
applyClone ( Entity clone , Boolean deepClone , Boolean ignoreEditorOnly )
nil
No description.
Parameters
clone
Entity
No description.
deepClone
Boolean
No description.
ignoreEditorOnly
Boolean
No description.
Render ()
nil
No description.
setStrokeColor ( Number r , Number g , Number b , Number a )
nil
Sets the color of the shape stroke if it's enabled.
Parameters
r
Number
Red value 0-1.
g
Number
Green value 0-1
b
Number
Blue value 0-1
a
Number
Alpha value 0-1 @see strokeEnabled
setStrokeWidth ( Number width )
nil
Sets the width of the shape stroke if it's enabled.
Parameters
width
Number
New stroke width. @see strokeEnabled
setGradient ( Number r1 , Number g1 , Number b1 , Number a1 , Number r2 , Number g2 , Number b2 , Number a2 )
nil
Colors the shape with a gradient. Radial for circles, linear for rectangles.
Parameters
r1
Number
New stroke width.
g1
Number
Red value of the first gradient color 0-1.
b1
Number
Green value of the first gradient color 0-1
a1
Number
Blue value of the first gradient color 0-1
r2
Number
Alpha value of the first gradient color 0-1
g2
Number
Red value of the second gradient color 0-1.
b2
Number
Green value of the second gradient color 0-1
a2
Number
Blue value of the second gradient color 0-1
clearGradient ()
nil
Removes the gradient from the shape.
getShapeType ()
Integer
No description.
setShapeType ( Integer type )
nil
No description.
Parameters
type
Integer
No description.
setShapeSize ( Number newWidth , Number newHeight )
nil
No description.
Parameters
newWidth
Number
No description.
newHeight
Number
No description.
buildShapeMesh ()
nil
No description.