My name is Luis Arriojas and lead the "Neo" team during the Koding's hackathon.
I am an U.S. citizen living in Venezuela and am trying to get a job after March 2015 in the San Francisco Bay Area.
My experience with the programming world comes from 9th grade on 1997 when registered myself in a program of Application Development at a local college called FundaUC, studying Turbo Pascal and FoxPro. After that, I learned by myself Visual Basic and Access. In 2005, I start working as a Web Developer using LAMP stack, and from 2012 use MEAN stack.
Now, I have a BS in CS and a MS in Web Engineering.
Its is a legal wizard to help users to understand TOS and similar documents. I chose to work under the theme #3 and called this project "Fluffy Lawyer":
"No one reads the fine print (ie TOS, EULA, legal documents) anymore yet every site has them. Devise a creative/interactive solution.
Why did I choose this name? Because it is funny.


Flash and JavaScript are required for this feature.
Download the video from iTunes U or the Internet Archive.
Description: We begin with a review of inference nets, then discuss how to use experimental data to develop a model, which can be used to perform simulations. If we have two competing models, we can use Bayes' rule to determine which is more likely to be accurate.
Instructor: Patrick H. Winston
Lecture 22: Probabilistic I...
Related Resources
Specifically developed for this lecture are the probabilistic inference notes (PDF), which are also listed among the readings for this course.
PROFESSOR PATRICK WINSTON: I was in Washington for most of the week prospecting for gold.
Another byproduct of that was that I forgot to arrange a substitute Bob Berwick for the Thursday recitations.

In any event, we have many explanations, none of them good.
But today we'll try to get back on track and you'll learn something fun.
In particular you will learn how a graduate student of mine Mark [? Phillipson ?], together with a summer UROP student, Brett van Zuiden, one of you-- managed to pull off a tour de force and recognize in these two descriptions the pattern that we humans commonly call "revenge." It was discovered.
The system didn't have a name for it, of course.
It just knew that there was a pattern there and sat waiting for us to give a name to it.
That's where we're going to end up.
But it'll be a bit of a journey before we get there, because we've got to go through all that stuff on the outline.
And in particular, we want to start off by a little tiny bit of review.
Because some of the stuff we did last time went by pretty fast.
In particular, you may remember they had this wonderful joint probability table, which tells us all we want to know, all we want to know.
We can decide what the probability of the police being called is given the this and the that, and all that sort of stuff, by clicking the appropriate boxes.
The trouble is, gee, there are only three variables there.
And when there are lots of variables it gets pretty hard to make up those numbers or to even collect them.
So we're driven to an alternative.
And we got to that alternative just at the end of the show a week ago.
And we got to the point where we were defining these inference nets, sometimes called "Bayes nets." And the one we worked with looked like this.
There's a burglar, a raccoon, the possibility of a dog barking, the police being called, and a trash can being overturned.
So more variables than that.
That only has three.
This has got five.
But we're able to do some magic with this because we, as humans, when we define-- when we draw this graph we're making an assertion about how things depend or don't depend on one another.
In particular, there's something to break down and memorize to the point where it rolls off your tongue.
And that is that any variable on this graph is said by me to be independent of any other non-descendant given its parents.
Independent of any non-descendant given its parents.
So that means that the probability of the dog barking, given its parents, doesn't depend on T, the trash can being overturned.
Because the intuition is all of the causality is flowing through the parents and can't get to this variable D without going through the parents.
So that is [? inserted ?] property of the nets that we draw.
And we tend to draw them in a way that reflects causality.
So it tends to make sense.
So somehow this thing is going to be-- we're going to use this thing instead of that thing.
But wait.
We may need that thing in order to do all the computations we want to perform.
So we need to be able to show that we can get to that thing by doing calculations on this thing.
So what to do?
Well, we're going to use the chain rule.
And remember that the chain rule came to us by way of the basic Axioms of Probability plus the definition plus a little colored chalk.
So we got to the point last time where we sort of believed this.
It's a really magical thing.
It says that the probability of all this stuff happening together is given as the product of a bunch of conditional probabilities.
And the conditional probabilities in this product are arranged such that this first guy depends on everybody else.
The second guy doesn't depend on the first guy but depends on everything else.
So that list of dependencies gets smaller and smaller as you go down here until it depends only one thing.
There's no conditional at all.
So that's going to come to our rescue because it enables us to go from calculations in here to that whole table.
But first I have to show you a little bit more slowly how that comes to be.
One thing I'm going to do before I think about probability is I'm going to make a linear list of all these variables.
And the way I'm going to make it is I'm going to chew away at those variables from the bottom.
I've taken advantage of a very important property of these nets.
And that is there no loops.
You can follow the arrows in any way so as you get back to yourself.
So there's always going to be a bottom.
So what I'm going to do is I'm going to say, well, there are two bottoms here, there's C and T. So I have a choice.
I'm going to choose C. So I'm going to take that off and pretend it's not there anymore.
Then I'm going to take this guy.
That's now a bottom because there's nothing below it.
I've already taken C out.
So we'll take that out next.
And now I've got this guy, this guy, and this guy.
This guy no longer has anything below it.
So I can list it next.
Now over here I've got raccoon and trashcan.
But trashcan is at the bottom.
So I've got to take it next because I'm working from the bottom up.
I want to ensure that there are no descendants before me in this list.
So finally I get to raccoon.
So the way I constructed this list like so ensures that this list arranges the elements so that for any particular element, none of its descendants appear to its left.
And now that's the magical order for which I want to use the chain rule.
So now I can write-- I can pick C to be my variable n.
And I can say that the chain rule says that the joint probability of all these variables P of C, D, B, T, and R-- the probability of any particular combination of those things is equal to the probability of C given everybody else.
Next in line is D given everybody else.
Next in line is T-- next in line is B given everybody else.
And next in line is T given everybody else.
And finally, just R. So this combination of things has a probability that is given by this chain rule expression.
Ah.
But first of all, none of those expressions condition any of the variables on anything other than non-descendants, all right?
That's just because of the way I've arranged the variables.
And I can always do that because are no loops.
I can always chew away at the bottom.
That ensures that whenever I write a variable, it's going to be conditioned on stuff other than its descendants.
So all of these variables in any of these conditional probabilities are non-descendants.
Oh wait.
When I drew this diagram, I asserted that no variable depends on any non-descendant given its parents.
So if I know the parents of a variable I know that the variable is independent of all other non-descendants.
All right?
Now I can start scratching stuff out.
Well, let's see.
I know that C, from my diagram, has only one parent, D. So given its parent, it's independent of all other non-descendants.
So I can scratch them out.
D he has two parents, B and R. But given that, I can scratch out any other non-descendant.
B is conditional on T and R. Ah, but B has no parent.
So it actually is independent of those two guys.
The trashcan, yeah, that's dependent on R. And R over here, the final thing in the chain, that's just a probability.
So now I have a way of calculating any entry in that table because any entry in that table is going to be some combination of values for all those variables.
Voila.
So anything I can do with a table, I can do in principle with this little network.
OK?
But now the question is, I've got some probabilities I'm going to have to figure out here.
So let me draw a slightly different version of it.
So up here we've got the a priori probability of B. Well, that's just probability of B. Down here with the dog, I've got a bigger table because I've got probabilities that depend on the values of its parents.
The probability of dog barking depends on the condition of the parents, nothing else.
So let's see.
I've got to have a column for B. I've got to have a column for the burglar and the raccoon.

But once I get those then I'll be able to calculate the probability of the dog barking.
So there are two of these variables.
So there are four combinations.
There's T T. There's T R, R T, and-- whoa, what am I doing?
Wake up!
T false.
False true.
And false false.
So what I really want to do is I want to calculate all of these probabilities that give the probability of the dog condition of the burglar and the raccoon.
Similarly, I want to calculate the probability of B happening doesn't depend on anything else.
So I don't know what to do.
Well, what I'm going to actually do is I'm going to do the same thing I had to do up there.

Maybe somebody hands it to me.
But I'm going to use that data to construct a bunch of tallies which are going to end up giving me the probabilities for all of those things.
So I don't know, let's see.
How should we start?
Step one, find colored chalk.
Step two, I'm going to extend these tables a little bit so I can keep track of the tallies.
So this is going to be all the ones that end up in a particular row.
And these are going to be the ones for which dog is true.
Similarly, I'm going to extend this guy up here in order to keep track of some tallies.
This is going to be the ones for which B is true.
And this one will be all.
So that's my set up.
And now suppose that my first experiment comes roaring in.
And it's all T's.
So I have T T T. That's my first experimental result, my first data item.
So let's see.
The arrangement here is burglar, raccoon, dog.
So burglar as a true.
And there's one tally count in there.
Likewise, the T T, that's the burglar and the raccoon, that brings me down to this first row.
So that gives me one tally in there and dog is true so that gives me a tick mark in that one.
All right?
Are you with me so far?
And now let's suppose that the next thing happens be all false.
Well, burglar is false.
But there is one experiment.
Everybody's false.
So we come down here to false false.
And that's the row we're going to work on.
We get a tally in there.
Do we put one in here?
No, because that's false.
Dog is false.
That's what our data element says.
So that's cool.
Maybe one more.
Let's suppose we have T T F. Well in that case, we have a tick mark here and a tick mark here because the burglar element is true.
Then we have T T. That brings us to the first row again.
So we get a tick mark there.
But dog as false, so no tick mark there.
That's how it works.
I suppose you'd like to see a demonstration, right?
Always like to see a demonstration.
So here's what it actually looks like.
So on the left you see the network as we've constructed it, with a bunch probabilities there.
And what I'm going to do now is I'm going to start simulating away so as to accumulate tick marks, tally marks, and see what kinds of probabilities that they indicate for the table.
I happen to be using a process for which the model on the left is a correct reflection.
So there's one simulation.
So the dog barking-- let's see, the burglar is false.
The raccoon is true.
I get one tick mark.
So the probability there is one.
Of course, I'm not going to just go with one.
I want to put a whole bunch of stuff in there.

No [? dice. ?] I don't even have an entry at all yet for T F here.
That's because I haven't run enough data.
So let me clear it instead of doing it one at a time.
Let me run 100 simulations.
See, it's still not too good.
Because it says this T T probability true.
This just because I'm feeding it data, right?
And I'm keeping track of what the data elements tell me about how frequently a particular combination appears.
Yes, [INAUDIBLE] STUDENT: So when you're doing one simulation, is that [INAUDIBLE] variables?
PROFESSOR PATRICK WINSTON: When I'm doing one simulation, I'm just keeping track of that combination in each of these tables.
Because it's going to tell me something about the probabilities that I want reflected in those tables.
So it's pretty easy to see when I go up here to burglar.
If I have a lot of data elements, they're all going to tell me something about the burglar as well as the other variables.
So if I just look at that burglar thing, the fraction of time that it turns out true over all the data elements is going to be its probability.
So now when I go down to the joint tables, I can still get these probability numbers.
But now they're conditioned on reticular condition of its parents.
So that's how I get these probabilities.
So I didn't do too well here because that T T combination gave me an excessively high probability.
So maybe 100 simulations isn't enough.
Let's run 10,000.
So with that much data running through, the probabilities I get-- let's see, I've got 893 here, instead of 0.9, 807 instead of 0.8, 607 instead of 0.6.
And that one's dead-on at 0.01.
So if I run enough of these simulations, I get a pretty good idea what the probabilities ought to be given that I've got a correct model.
OK, so that takes care of that one.
And of course, I didn't draw the other things in here.
But by extension, you can see how those would work.
Oh.
But you know what?
I think I will put a little probability of raccoon table in here.
Because the next thing I want to do is I want to go the other way.
This is recoding tallies from some process so I can develop a model.
But once I've got these probabilities, of course, then I can start to simulate what the model would do.
All right?
How would I do that?
Well, do I want to use the same table?
I think just to keep things sanitary, what I'll do is I'll go over here and do it again.
Here's B. It's got a probability of B. Here's R.
Here's a table probability of R. That comes down into a joint table for dog.
And it's got four elements.
Depending on the burglar condition and the raccoon condition, we get a probability of dog.
And now, imagine these have all been filled in.
So what do I want to do if I want to simulate this system generating some combination of values for all the variables?
Well, I do the opposite of what I did when I was working around with this chain rule showing that I could go from the table to those probabilities.
Now I've got the probabilities.
I'm going to go the other direction.
Instead of chewing away from the bottom, I'm going to chew away from the top.
Because when I go into the top and chew way, everything I need to know to do a coin flip is there.
So in particular, when I go up in here, I've got the probability of burglar now.
So I'm going to use that probability to flip a coin.
Say it produces a T. So that takes care of this guy.
And I can now scratch it off since it's no longer in consideration.
It's no longer a top variable.
So now I go over into raccoon and I do the same thing.
I take this probability.
I do a flip.
And say it produces an F.
But now, having dealt with these two guys, that uncovers this dog thing.
And I've got enough information, because I've done everything above, to make the calculation for whether to dog is going to be barking or not.
But wait.
I have to know that I've got a T and a T and a T and an F and an F and a T and an F and an F. Because I have to select the right row.
So I know that B is T. And I know that R is F. So that takes me into the table into the second row.
So now I get this probability.
I flip that coin and I get some result, say, T. Voila.
I can do that with the other two variables.
And I've got myself an experimental trial that is produced in accordance with the probabilities of the table.
OK?
Of course-- yeah, in fact, how did I get those numbers?
Actually what I did is I used the model on the left to generate the samples that were used to compute the probabilities on the right.
So you've seen that a demonstration of this already.
Now of course-- I don't know, all of this sort of depends on having everything right.
I've written a thing to write it one more time.
Burglar, raccoon, dog, call the police, trashcan.
But somebody else may say, oh, you've got it all wrong.
This is what it really looks like.
The dog doesn't care about the raccoon at all.
So that's a correct model.
Now when I do a simulation, I could fill in the tables in either model, right?
I'm sure you'd like to see a demonstration.
So let me show you a demonstration of that.
So there are the two tables.
And I can run 10,000 simulations on those guys, too.
Now, look.
The guy on the left is a pretty good reflection of the probabilities in a model I used to produce the data.
But the guy on the right doesn't know any better. it just fills in its own tables, too.
So what to do?
I say this one's the right model.
And you say that one's the right model.
Who's right?
Maybe we'll never know.
And the guy on the left will get rich in the stock market and the guy on the right will go broke.
I would be nice if we could actually figure out who's right.
So would you to see how to figure out who's right?
Yeah, so would I. What we're going to do is we're going to look at naive Bayesian inference.
And that's our next chore.
So here's how it works.
We know, from the definition of conditional probability, we know that the probability of A given B is equal to the probability of A and B divided by the probability of B, right?
Equal to by definition.
So that means that the probability of A given B times the probability of B-- I'm just multiplying it out-- it equal to that joint probability.
Oh, but by symmetry, there's no harm in saying I can turn that around and say that the probability of B given A times the probability of B is also equal to that joint probability, right?
I've just expanded it a different and symmetric way.
If I've got to write a, b on B, b, a on A. Thank you.
Who was complaining?
Good work.
That would have been a major-league disaster.
But now, having written that, I can forget about the middle.
Because all I'm really interested in is how I've turned the probabilities around in that conditional.
Why would I care about doing that?
By the way, we're now talking about the work of the Reverend Bayes.
Because we can rewrite this yet again as the probability of A given B is equal to the probability of B given A times the probability of A divided by the probability of B.
That's just elementary algebra.
But now I'm going to do something magical.
I'm going to say I've got a classification problem.
I want to know which disease you have.
That's a classification problem.
Maybe you've got the swine flu.
Maybe you've got indigestion.
Who knows.
But I get all these symptoms.
I get all these pieces of evidence.
You've got a fever.
You're throwing-- oh, well, let's not go into too much detail, there.
But what I'm going to do is I'm going to say, well, let's suppose that A is equal to a class that I'm interested in, the disease you've got.
And B is equal to the evidence, the symptoms I observe.
Voila.
I may have a pretty hard time figuring out what the probability of the class is given the evidence.
But figuring out the probability of the evidence given the class might not be so hard.
Let me get another board in play and show you what I mean.
By plugging class and evidence into Bayes' rule, what I get is the probability of some class given the evidence is equal to the probability of the evidence given the class times the probability of the class divided by the probability of the evidence.
Now you've got to let that sing to you a little bit.
Suppose I've got several classes that I'm trying to decide between.
I'm trying to select the best out of that batch of classes.
Well, I've got the evidence.
And if I know the probability of the evidence given each of those classes, and if I know, a priori, the initial probability the class, then I'm done.
Because I've got the two elements in the numerator.
Why am I done?
Because the denominator is the same for all the classes.
It's just the probability of the evidence.
And then I could just sum everything up.
I know it adds to 1 anyway.
So that's cool.
But sometimes there's evidence-- actually there's more than one piece of evidence.
Let's say that there's some class.
some i, and we're trying to figure out if that's the correct class.
So we've got c sub i there and c sub i there.

So it could be e sub 1, e sub n, oops, premature right bracket.
All that evidence, given the class i times the probability of the class i over some denominator that we don't care about because it's going to be the same for everybody.
So we'll just write that as d.
Now what if these pieces of evidence are all independent given the class?
So if you have the swine flu, the probability you have a fever is independent of the probability you're going to throw up, say.
Then can we write this another way?
An easier way?
Sure.
Because when things are independent, the joint probability is equal to the product of the individual probabilities.
So that is to say-- it's easier to see it if you write it down than if you just say it-- this probability here from these two elements here is equal to the probability of e sub 1 conditioned on c sub i times the probability of e sub 2 conditioned on c sub i, all the way down to the probability of e sub n conditioned on c sub i divided by some denominator we don't care about.
See, what I'm going to try to do is I'm going to go through this for all the ci and see which one's the biggest.
STUDENT: That's the [INAUDIBLE] ci, right?
PROFESSOR PATRICK WINSTON: This is the probability of-- STUDENT: [INAUDIBLE] right-hand side [INAUDIBLE].
PROFESSOR PATRICK WINSTON: Right here?
Oh yes, you're quite right.
Oh yeah, thanks.
I can't write and think at the same time.
Thanks.
OK.
So I've just figure out which one of these is the biggest.
And I've identified the class.
Now you say to me, well, I would like to see an example.
So-- I don't know, does anyone have any spare change?
A nickel, a quarter.
This is not because of infinitesimally low raises here at MIT.
I just need it for a demonstration.
I need two coins.
Don't forget to get these back, I tend to be-- Now suppose these two coins are not exactly the same.
One of these points is a legitimate, highly-prized American quarter.
The other one is a fake.
And with this one, the probability of heads, let us say, is 0.8 instead of 0.5.
So I mix these all up.
And I pick one.
And I start flipping it.

Then I flip it again.
And I get a tail.
Which coin did I pick?
Well, we're going to use this stuff to figure it out.
Here's what happens.
Before I forget.
Thank you very much.
So what we've done is we've selected these things from my hands.
And I can't draw hands.
So I'll draw a little cup here.
And there are two coins in here.
And we're going to pick one.
And one has a probability of heads equal to 0.8.
And this one has a probability of a head of 0.5.
So here's the draw.
I pick one.
Each has a probability of 0.5.
This one is the one with the 0.8 as the probability of head.
And this one is the one with the probability of 0.5 as a head.
OK?
So now suppose the first flips as it was is T. Well, that's a piece of evidence.
That's here.
Probably of evidence given the class.
Well in the case of having drawn this biased coin, the probability of coming up with a tail-- ah, let's say a head, just to make my numbers a little easier.
Probability of coming out there with a head is equal 0.8 given that it's up here in this choice.
The probability given that you have a fair coin is 0.5.
So now if we take the next coin and take it to be a tail then the probability of this guy given that evidence is 0.2.
And the probability of this guy given that evidence-- it's a fair coin, so it doesn't care.
It's still 0.5.
So now what's the probability of this class given this evidence?
It's the product 0.5 times 0.8 times 0.2.
And what's the probability of this guy?
It's 05 times 0.5 times 0.5, divided by a denominator which is the same in both cases.
So let's forget about this early 0.5 here.
Because it's the same in both cases.
And we just multiply those numbers together.
That gives us 0.8 times 0.2.
What's that?
0.16?
And this guy, 0.5 times 0.5, that's 0.25.
So it looks an awful lot like-- with this combination-- that I've picked the coin that's fair.
One more flip?

So that puts a 0.8 in here.
And 0.5 in here.
When you multiply those out that's 0.125.
And this is 0.128.
So it's about equal.
So you see how that works?
All right.
So we're using the coin flips as evidence to figure out which class is involved.
OK so I don't know, you'd probably like to see a demonstration of this, too, right?
You say to me, gosh, just two kinds of coins.
That's not very interesting.
Let's try five kinds of coins.
So what I want to show you is how the probabilities for all these coins-- there are five of them, color-coded-- how the probabilities vary with a series of flips.
Let's suppose I've got a head-- the grey line, by the way, is the fraction of heads-- so that's going to be one.
Because I'm just doing heads.
You see that black line rising?

That's the probability that the-- that's the coin which only shows heads, the probability of head is 1.
And I'm flipping a whole bunch of heads here.
Isn't that cool?
Now what happens if I suddenly put in a tail?
By the way, you'll no doubt, here one the extreme left-- the initial probability of the P=0 coin was 0.1.
As soon as I flipped a head that went to 0.
And it will never get off 0, right?
That makes sense.
Because if the probability that you'll get a head is 1 you should never see a tail.
If you ever do, that isn't your coin.
What happens now if I interrupt a series of heads and produce a tail?
STUDENT: [INAUDIBLE].
PROFESSOR PATRICK WINSTON: What's that?
STUDENT: [INAUDIBLE].
PROFESSOR PATRICK WINSTON: The black one will go to 0.
What else happens?
By the way, the blue one is the one with the highest probability of being a head.
[INAUDIBLE] Boom!
That blue one shot up.
Not going up slowly.
It shot up.
Because now the preponderance of evidence with all those heads is that I've flipped the coin with a bias of 0.75 towards heads.
So let's clear this.
Pick any probability you want.
0.25, 0.5, and so on.
I don't know, let's pick 0.25 since we've been at the upper end.
So orange is 0.25.
And sure enough, the probability that I've selected the 0.5 coin is going up and up and up and up after the original irregularity.
The Law of Large Numbers is setting in.
And a probability that I've got that 0.25 coin in play is pretty close to 1.
All right.
So that's cool.
Now you say to me, that's awfully nice but stop.
Awfully nice, but not very real-world-ish.
So let me give you another problem.
It's well-known that you are, with high probability, of the same political persuasion as your parents.
So if I wanted to figure out which party a parent belongs to, I could look at the party that their children belong to, right?
So it's just like flipping coins.
The particular coin I have chosen corresponds to the parent.
Individual flips correspond to the political party that the child belongs to.
So let's get up a little bit-- by the way, I wrote all this stuff over the weekend.
So who knows if any of it will work.
But let's see.
A parent party classifier.
There it is, Democrats and Republicans.
And now the prior for being a Republican given here is 0.5.
But I don't know, this is a little bit Democratic state.
So let's adjust that down a little bit.
Somewhere in there might be about right But let's just, for the sake of a classroom illustration, go down here.
So now the meter is showing the prior probability because that's the only thing in the formula so far.
I've got no evidence.
So now let's suppose that child number one is a Republican.
Back to neutral.
So I've got a low probability that the parent-- a priori probability that the parent is a Republican and a child who's a Republican.
I notice that 0.2 and 0.8, the conditional is 0.8.
And the prior is 0.2.
That's why it comes out to balance each other, right?
So now if we get another Republican in there it goes way up.
If I have a Democratic child it goes back down.
If I have an equal balance between children then it goes way back down because of that prior probability being low.
So if I make that high, even though the children are balanced, I'm still going to have a high probability of being a Republican.
Now let's see.
If I take that slider there, the conditional probability, and drive it to the left here-- let me make that equally in.
And let's make that one thing.
I don't know.
What am I doing now?
If I make the probability less than 0.5, what's that mean?
That means you're sore at your parents and you want to belong to a different party.
All right, so now, what's next?
Oh gosh.
What's next?
This is what's next.
What's next to somewhere?
Yeah, this is what's next.
This here.
We've got two models.
Remember when I said we wanted to decide between them?
Can we use that Bayesian hack to do that, too?
Sure.
Because we've got these two models.
We've got the probabilities in them.
So now I can take my data and calculate the probability of a left model given the data and the probability of the right model given the data, multiply that times their a priori probabilities, which I'll assume are equal.
Then I can do a model selection deal much in defiance to what I was hinting at before.
so let's try that.
Whoa.
There are my two models.
Yes, there they are.
We've already trained them up.
And they've got their probabilities.
Now what we're going to do is we're going to use the original model to simulate the data.
So what we're going to do is we're going to simulate draws, simulate events, similarly combinations of all variables using a model that looks like the one on the left, that is the one on the left except for the slight differences in probabilities, OK?
Then we're going to do this Bayesian thing and see where the meter goes.
So we'll run one data point.
Oops, went the wrong way.
Makes me nervous.
I just finished this at 9:15.
Maybe there's a bug.
Oops, two data points, swings to the left.
Three data points, back to the right.
Of course that's not much data.
So let's put some more data in.
Yeah.
Boom, there it goes.
Let's try that again.
That was cool.
So let's run 1,000 simulations and one data point.
It bobbles around a little bit and goes flat over to the left.
Because that is the model that reflects the one that the data is generated from.
So now we got Bayesian classification, except now the classification has gone one step more and it becomes structure discovery.
We've got two choices of structure.
And we can use this Bayesian thing to decide which of the two structures is best.
Isn't that cool?
Well, it's only cool if you could do what?
So if you had two choices-- you can select between them and pick the best one-- but there are-- gosh, for this number of variables, there are a whole lot of different networks that satisfy the no looping criteria and don't have very many parents.
There's an awful lot of them.
In fact, if you strict this network to two parents there are probably thousands and thousands of possible structures.
So do I try them all?
Probably not.
It's too much work when you get 30 variables or something like that.
So what do you do?
We know what to do, right?
We're almost veterans a 6034.
We have to search!
So what we do is we take the loser and we modified it.
And then we modify it again.
And we keep modifying it until we drop dead or we get something that we're happy with.
So let's see what happens if we change this problem a little bit and do structure discover.
We're starting out with nothing linked.
And we're going to just start running this guy.
So what's going to happen is that the good guy will prevail.
And the bad guy will be a copy of the good guy perturbed in some way.
So it's a random search.
You'll notice that score-- it's too small for you to read.
All these things are too small to read.
Let me make it a little bigger.
Too small to read, but that number on the right there is not the product of the probabilities, actually.
It's the sum of the logarithms of the probabilities.
They go together, right?
And the reason you use this instead of the probabilities is because these numbers get so small that was a 32-bit machine, you eventually lose.
So use the log of the probabilities rather than the product of the probabilities.
You use the sum of the logs instead of the product of the probabilities.
And eventually, you hope that this thing converges on the correct interpretation.
But you know what?
This thing is so flat as a space and so a large and so telephone pole-like that it's full of local maxima.
So what this program is doing is every once in awhile-- I think with probability 1 and 10; I forgot what parameters I used-- every once in awhile, it'll do a total radical rearrangement of the structures.
In other words, it's a random restart.
It keeps track of the best guy so far.
And every once in awhile it does a totally random restart in its effort to search the space.
So that's how you go from probabilistic inference to structure discovery.
Now when is this stuff actually useful?
Well, I hinted at a medical diagnosis, right?
That's a situation where you've got some symptoms.
And you want to know what the disease is.
So as soon as you use the keyword "diagnosis," you've got a problem for which this stuff is a candidate.
So what other kinds of diagnosis problems are there?
Well, you might be lying to me.
So I can put a lie detector on you.
And each of those variables that are measured by the lie detector are an independent indication whether you're telling the truth or not.
So it's this kind of Bayesian discovery thing.
Naive Bayesian Classification.
What other kinds of problems speak to the issue of diagnosis?
Well, we like to know how well you know the material!
So we can use quizzes as pieces of evidence.
Thank god we don't use exactly a naive Bayesian classifier, because then we wouldn't be able to do that combination.
We have to use a slightly more complex-- what you can think of as a slightly more complex Bayesian net to do that particular kind of diagnosis.

You're trying to figure out what to do next, what the cause is.
So using the evidence to go backward to the cause.
So maybe you've got some program that doesn't work.
Happens to me a lot.
So I use the evidence from the symptoms of the misbehavior to figure out what the most probable cause is.
But now to conclude the day-- last time there weren't any powerful ideas.
But if you take the combination of the last lecture and this lecture to be a candidate for gold star ideas, these are the ones I'd like to leave you with.
We got here is-- this Bayesian stuff, all these probabilistic calculations are the right thing to do.
They're the right way to work when you don't know anything, which would make it sound like you're not very useful, because you think you always-- well, in fact, there are a lot of situations where you either can't know everything, don't have time to know everything, or don't want to take the effort to know everything.
So in medical diagnosis all you've got is the symptoms.
You can't go in there and figure out in a more precise way exactly what's wrong.
So you use the symptoms to determine what the cause is.
And then all those other kinds of cases that I mentioned.
But now, what other kinds of structure discovery are there?
Well, the kind of structure discovery that I hinted at in the beginning will be the subject that we'll begin with during our next and sadly final conversation here in [? 10250 ?] on Wednesday.
It will feature not only a discussion of how this stuff can be used to discover patterns and stories, but we'll also talk about what's on the final, what kind of thing you could do next, that sort of thing to finish off the subject.
And that's the end of the story for today.


Integrating with Other Libraries
Because of its tiny size and modular nature, it's easy to use with many other third-party libraries and frameworks. You can use Material Design Lite, Bootstrap, jQuery, Redux, Mobx, Lodash, Ramda, etc.
The only thing to be aware of is markup. If you are using JSX, any markup must respect the rule of well-formedness. This means that all HTML self-closing tags will have to be escaped in the render function with a forward slash:
wrong correct
----------------------

<input> <input/>
<col> <col/>
<param> <param/>
<link> <link/>
<meta> <meta/>
Material Design Lite
Material Design Lite is a popular frame that enbles you to use Google Material Design styles and controls from Android as the basis for a Web application. Implementation is really easy. Just include the Material Design Lite resources in out project and import them in your index.html
file. Or you could just load them from a CDN.
Here's an example of a Hello World using MDL. It has a dropdown menu in the right:
See the Pen Composi MDL-2 by Robert Biggs (@rbiggs) on CodePen.
Literally, it's just a question of using the Material Design Lite classes on the markup that a component creates for Material Design Lite to initialize dynamic components. Since we've imported the resources to implement a Material Design dialog in our index.html
file, let's see how to use it. We'll make a component that creates a Material Design Lite card that has a button to show a dialog. The MDL dialogs work, the markup uses the HTML tag dialog
. This is not supported by older browsers, hence our need for the dialog polyfill that we have inported above. The dialog itself is static markup for now. We output it with the card. The card button will show the dialog:
See the Pen Composi MDL-3 by Robert Biggs (@rbiggs) on CodePen.
And, here is an example of some form inputs and a list:
See the Pen Composi MDL-4 by Robert Biggs (@rbiggs) on CodePen.
Bootstrap
Now let's see how to use Bootstrap with Composi. Like we did with Material Design Lite, we need to import the Bootstrap resources into our index.html
file.
Here we have a Jumbotron with a drop down menu. Notice how we had to unwrap the props in the end. That's because each time you nest a function component using JSX that passes props down, the props get wrapped in another enclosing object. Because we have our Bootstrap resources load, we can just output this markup from our component:
See the Pen Composi Bootstrap by Robert Biggs (@rbiggs) on CodePen.
jQuery
In our example with Bootstrap we already demonstrated using a jQuery plugin with Composi. It's that simple.
A jQuery plugin can be used on the same page as Composi, and even inside a Composi component, as we did with Bootstrap, as long as the plugin is not going to modify the DOM structure. Remember, it's Composi's job to manage your app's DOM. If you need to use a jQuery plugin that does manage the DOM structure, use it outside of your Composi components.
Communicating between jQuery and Composi
You might have a jQuery plugin whose state you want to track in a Composi component, or vise-a-versa. Depending on the API of the plugin, you might be able to do this by using a pubsub library. These are tiny and powerful for communication between disparate components.
Redux & Mobx
We already have a tutorial on how to use Redux and Mobx with Composi.


wTrans
Web based transcription tool
Upload an audio
Find silences
Export as CSV
Help
/
There's no transcription yet, select a segment to transcript
Speakers list
VOS
VAS
SRT
NSI
ESIg
MRG
MRGg
LRS
LAS
LAG


The Open Database Connectivity (ODBC) interface allows applications to access data from database management systems (DBMS), using Structured Query Language (SQL) expressions.
J Win9x/NT only supports 32-bit ODBC drivers.
The J/ODBC interface uses the Data Driver verbs, defined in script dd.ijs.
The processing required by the application is essentially independent of the DBMS. For example, if you have written programs to access dBase files using ODBC, then you can use similar programs to access Paradox, or indeed any other DBMS.
The application, and the driver programs that access the DBMS, are physically separate. To access a DBMS, you need only ensure a driver for that DBMS is available. There are ODBC drivers for virtually all commercial DBMS, and J itself is distributed with drivers for several popular systems, including Access, Btrieve, dBase, Excel, FoxPro, Oracle, Paradox and SQL Server.
Note that the DBMS need not itself support SQL, for example dBase does not. All that matters is that the ODBC DBMS driver is available.
An ODBC DBMS driver, together with information on where its datasets are stored, is typically referred to as a data source.
Between the application and the data there are 3 layers, though they appear to the application as a single unit:
J application | ||
Data Driver | ||
ODBC Manager | ||
Data Source | Data Source | Data Source |
Data | Data | Data |
The application sends requests to the Data Driver (using the verbs defined in script dd.ijs), which converts them into a standard format and sends them to the ODBC Manager. The ODBC Manager then sends them to the appropriate Data Source, and is also responsible for ensuring the required drivers are loaded. The Data Source drivers then handle the data access.


|
|
P
|
R
|
O
|
G
|
R
|
A
|
M
|
M
|
I
|
N
|
G
|
|
C
|
O
|
M
|
P
|
E
|
T
|
I
|
T
|
I
|
O
|
N
|
|
Job
Processing
|
|
|
|
Task Description
A factory
is running a production line. Two operations have to be performed on each job:
first operation "A", then operation "B". There is a certain number of machines
capable of performing each operation.
|
|
Source: 8th International Olympiad in Informatics, Veszprém,
Hungary
http://www.inf.bme.hu/contests/tasks/ioi96d1.html
Program Input
The
input file contains positive integers in five lines. The first line contains
N, the number of jobs (1<=N<=1000). On the second line, the number M1 of type
"A" machines (1<=M1<=30) is given. In the third line there are M1 integers, the
job processing times of each type "A" machine. The fourth and the fifth line
contain the number M2 of type "B" machines (1<=M2<=30) and the job processing
times of each type "B" machine, respectively. The job processing time is measured
in units of time, which includes the time needed for taking a job from a container
before processing and putting it into a container after processing. Each processing
time is at least 1 and at most 20.
5
2
1 1
3
3 1 4
Program Output
Your
program should write two lines to the output file. The first line should contain
the solution of subtask A. The second line should contain the solution of subtask
B. The program must write the sentence labels as shown in the example below:
Minimum time to complete subtask A: 3
Minimum time to complete subtask B: 5


EventFinder
Where are you?
X
-10
-9
-8
-7
-6
-5
-4
-3
-2
-1
0
1
2
3
4
5
6
7
8
9
10
Y
-10
-9
-8
-7
-6
-5
-4
-3
-2
-1
0
1
2
3
4
5
6
7
8
9
10
Submit
Find my nearest events!


Corpus Topical Structure: Network Analysis and Topic Modeling
In this post, I will be reviewing two principal ways to uncover the topical structure of texts: community detection algorithms on networks of words and topic models. The data I use come from the Web of Science. I have records for all articles containing “bioethic” in their titles, abstracts, or keywords, and the time period ranges from 1900-(Jan.)2018. There are roughly 8,500 articles in this data set (I am happy to provide a much more detailed description upon request).
library(tidytext)
library(readxl)
library(dplyr)
library(stringr)
library(ggplot2)
library(tidyr)
library(igraph)
library(reshape2)
library(tidygraph)
library(stm)
Community Detection
In order to run topic models with the article’s abstracts, it is necessary to know k, the number of topics. However, this value is unknown, and there is no non-arbitrary number or range with which to begin. As such, I will start with the number of communities identified from the keyword-keyword network (network
), which is a single mode weighted network.
# louvain algorithm
comm_louvain <- cluster_louvain(network)
# label propagation algorithm
comm_label_prop <- cluster_label_prop(network)
#membership(net.comm)
#communities(net.comm)
#crossing(net.comm, network)
#algorithm(net.comm)
#is_hierarchical(net.comm)
#plot(as.dendrogram(net.comm))
The estimates for community number vary quite widely, but, given that only the louvain (63) and label propagation (85) results present a realistic task, I will use the ldatuning package to check fit measures where k varies between 60 and 90. Note: If you would like a detailed comparison of the igraph community detection methods and their relative performances on difference types of networks, I suggest this; the other methods produced wildly divergent estimates for the number of communities (800-3000).
However, before moving further, let’s check out the communities themselves. I first examine the community igraph object obtained with the louvain algorithm. Printing the object gives a preview of the keywords in the first community and indicates that we have 63 groups and a graph modularity of .47. The next function sizes()
presents a three-row table; within each row, the top cell indicates the community id and the bottom cell indicates the number of words within it. Finally, return to the short list of the first community members. There obviously seems to be an issue with this method (at least for the first community) because most of the keywords do not intuitively belong together.
# print basic summary
comm_louvain
## IGRAPH clustering multi level, groups: 63, mod: 0.47
## + groups:
## $`1`
## [1] "institutional" "community"
## [3] "personalized" "evidence"
## [5] "review" "based"
## [7] "regenerative" "systematic"
## [9] "precautionary" "emergency"
## [11] "literature" "medicine"
## [13] "principle" "peer"
## [15] "forensic" "alternative"
## [17] "precision" "board"
## + ... omitted several groups/vertices
# size of each community
sizes(comm_louvain)
## Community sizes
## 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
## 227 232 207 261 7 3 389 38 6 15 320 458 234 321 4 76 10 3
## 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36
## 16 29 570 14 341 7 2 39 2 2 2 2 2 2 2 2 4 5
## 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54
## 86 4 2 2 2 161 2 5 115 4 150 3 4 2 2 2 6 2
## 55 56 57 58 59 60 61 62 63
## 119 5 2 2 225 2 2 143 2
The igraph package makes it easy to visually inspect the sizes of the communities. In the graph using the plot()
function, we get a graph in which the x axis lists the communities and the y axis reflects their sizes. This presentation does not easily convey results; the second graph is a histogram. Between the two, it should be relatively easy to see that the vast majority of communities are rather small, but there are a few containing many keywords. I will interpret this finding shortly below, after I get the same information for the other community object.
plot(sizes(comm_louvain))
hist(sizes(comm_louvain))
Let’s replicate the above examination on the label propagation community object. We arrive at 73 communities with this algorithm, but the modularity is drastically lower at .059! We should be skeptical moving forward because there is certainly more conceptual coherence within the bioethics literature. The largest community has 4,609 members, which is problematic: there is likely more nuance within any given subfield of bioethics than that. Further, so many isolated communities with 2-5 members does not correspond to intuitive sense about the field. I will not bother visualizing these results. Do note though that there is at least a bit more coherence in the group of keywords. For example, the words “informed”, “consent”, and “assent” clearly go together, and words, such as “forms” and “psychoses”, relate to the application of this general topic to patient treatment and the process of gaining consent.
# print basic summary
comm_label_prop
## IGRAPH clustering label propagation, groups: 80, mod: 0.04
## + groups:
## $`1`
## [1] "informed" "health"
## [3] "public" "stem"
## [5] "human" "clinical"
## [7] "cell" "palliative"
## [9] "medical" "genetic"
## [11] "organ" "brain"
## [13] "life" "qualitative"
## [15] "advance" "embryonic"
## [17] "assisted" "hiv"
## + ... omitted several groups/vertices
# size of each community
sizes(comm_label_prop)
## Community sizes
## 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
## 4602 8 7 9 6 7 4 3 5 2 5 2 2 4 2
## 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
## 3 2 4 3 2 12 4 8 2 5 8 7 3 3 2
## 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45
## 2 3 4 9 2 6 2 2 2 5 2 12 5 2 6
## 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
## 2 4 4 4 5 4 6 4 2 3 6 2 2 2 5
## 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75
## 5 5 4 2 3 3 3 3 4 3 2 2 2 2 2
## 76 77 78 79 80
## 2 2 2 2 2
At this point, things seem pretty bleak for the community detection algorithms, so we definitely want to check out the topic modeling approaches. However, before we do that, let’s use a different network: one constructed from abstracts instead of keywords. This obviously will change the substantive interpretation of the community structure, but perhaps the keyword network data were simply poorly structured for community detection.1
The following code create a network in which words within abstracts are connected when they appear within the same abstract and have weights which correspond to the number of abstracts in which they co-occur.
abstract_bigrams <- data.bioeth.1 %>%
filter(!is.na(abstract)) %>%
unnest_tokens(bigram,
abstract,
token = "ngrams",
n = 2) %>%
separate(bigram,
c("word1", "word2"),
sep = " ") %>%
filter(!word1 %in% stop_words$word) %>%
filter(!word2 %in% stop_words$word) %>%
filter(word1 != "bioethics"
& word1 != "ethics") %>%
filter(word2 != "bioethics"
& word2 != "ethics")
# create weighted edgelist
ab_edges_w <- abstract_bigrams %>%
count(word1, word2, sort = TRUE)
# create network
ab_network <- ab_edges_w %>%
rename(weight = n) %>%
graph_from_data_frame(directed = FALSE)
# verify that edge weights are correct
#setequal(ab_edges_w$n, E(ab_network)$weight)
Now, on to community detection. I will not spend time describing the network here because that is a bit tangential to the purpose of this document.
# louvain algorithm
comm_louvain_ab <- cluster_louvain(ab_network)
# label propagation algorithm
comm_label_prop_ab <- cluster_label_prop(ab_network)
How do the community structures within these objects differ from the previous two? I will begin as before with the community object based upon the louvain algorithm. Now, we have 300 communities with a modularity of 0.38. We can see from the sizes()
function that there exists a very large range of community sizes. The first histogram presented illustrates the massive positive skew, and the second histogram, in which I set the maximum x value to 1000, presents the same problem for interpretability.  We observed a large modularity difference between the two algorithms in the keyword networks, as well. The largest community contains almost 22,000 members, but there seems to be more variation among the other community sizes. The majority of the remaining communities are not larger than 400 words, so I will truncate the x axis of the following histogram to help (hopefully) with interpretation. It appears that most communities have below 100 members, but there is a non-negligible contingent that contain a few hundred members.
# general info
comm_label_prop_ab
## IGRAPH clustering label propagation, groups: 413, mod: 0.03
## + groups:
## $`1`
## [1] "health"
## [2] "ethical"
## [3] "public"
## [4] "human"
## [5] "stem"
## [6] "bioethical"
## [7] "rights"
## [8] "clinical"
## [9] "cell"
## + ... omitted several groups/vertices
# community sizes
sizes(comm_label_prop_ab)
## Community sizes
## 1 2 3 4 5 6 7 8 9 10 11 12
## 21841 28 11 6 20 2 3 231 14 6 6 5
## 13 14 15 16 17 18 19 20 21 22 23 24
## 5 4 2 2 4 4 4 11 7 3 26 10
## 25 26 27 28 29 30 31 32 33 34 35 36
## 29 2 6 4 2 2 2 4 14 2 4 6
## 37 38 39 40 41 42 43 44 45 46 47 48
## 2 33 4 15 3 3 3 2 2 2 2 6
## 49 50 51 52 53 54 55 56 57 58 59 60
## 2 2 2 5 2 6 2 13 3 2 2 5
## 61 62 63 64 65 66 67 68 69 70 71 72
## 19 2 2 43 2 3 2 5 2 2 2 2
## 73 74 75 76 77 78 79 80 81 82 83 84
## 3 3 2 2 6 2 5 3 4 2 2 2
## 85 86 87 88 89 90 91 92 93 94 95 96
## 4 3 2 2 2 4 7 3 3 2 8 4
## 97 98 99 100 101 102 103 104 105 106 107 108
## 4 4 3 2 2 2 2 3 4 15 3 3
## 109 110 111 112 113 114 115 116 117 118 119 120
## 4 4 2 2 2 2 3 2 3 2 2 3
## 121 122 123 124 125 126 127 128 129 130 131 132
## 3 2 3 2 2 2 2 10 2 2 2 2
## 133 134 135 136 137 138 139 140 141 142 143 144
## 2 2 2 2 3 2 2 3 2 2 7 2
## 145 146 147 148 149 150 151 152 153 154 155 156
## 2 2 3 7 5 2 2 2 2 3 2 2
## 157 158 159 160 161 162 163 164 165 166 167 168
## 2 4 3 2 2 3 2 5 3 2 2 2
## 169 170 171 172 173 174 175 176 177 178 179 180
## 2 2 2 2 5 4 2 4 3 2 2 3
## 181 182 183 184 185 186 187 188 189 190 191 192
## 2 2 4 2 2 2 2 5 3 2 2 2
## 193 194 195 196 197 198 199 200 201 202 203 204
## 2 4 2 2 2 2 2 7 2 3 2 2
## 205 206 207 208 209 210 211 212 213 214 215 216
## 2 2 2 2 3 4 3 4 2 2 2 2
## 217 218 219 220 221 222 223 224 225 226 227 228
## 2 3 2 2 2 2 2 10 2 2 3 2
## 229 230 231 232 233 234 235 236 237 238 239 240
## 2 2 3 2 2 2 2 3 3 4 3 2
## 241 242 243 244 245 246 247 248 249 250 251 252
## 2 2 2 2 2 2 2 2 2 2 2 2
## 253 254 255 256 257 258 259 260 261 262 263 264
## 4 2 2 2 2 2 2 2 2 2 2 2
## 265 266 267 268 269 270 271 272 273 274 275 276
## 4 2 6 2 2 2 2 3 2 2 2 2
## 277 278 279 280 281 282 283 284 285 286 287 288
## 3 2 2 2 2 2 2 2 2 3 3 2
## 289 290 291 292 293 294 295 296 297 298 299 300
## 2 5 2 3 2 3 2 2 2 2 2 2
## 301 302 303 304 305 306 307 308 309 310 311 312
## 2 2 3 2 2 2 2 2 2 3 2 2
## 313 314 315 316 317 318 319 320 321 322 323 324
## 2 2 1 2 2 2 2 2 2 2 2 6
## 325 326 327 328 329 330 331 332 333 334 335 336
## 2 2 2 2 2 2 3 2 2 2 2 2
## 337 338 339 340 341 342 343 344 345 346 347 348
## 2 2 2 3 2 2 2 2 2 2 2 2
## 349 350 351 352 353 354 355 356 357 358 359 360
## 2 3 2 2 2 2 2 2 2 2 1 2
## 361 362 363 364 365 366 367 368 369 370 371 372
## 3 2 2 2 2 2 2 2 2 2 2 2
## 373 374 375 376 377 378 379 380 381 382 383 384
## 2 2 2 2 2 2 2 2 3 2 2 2
## 385 386 387 388 389 390 391 392 393 394 395 396
## 2 2 2 2 2 2 2 2 2 2 2 3
## 397 398 399 400 401 402 403 404 405 406 407 408
## 1 2 2 2 2 2 2 2 2 2 2 2
## 409 410 411 412 413
## 2 2 2 2 2
# plot size distribution
hist(sizes(comm_louvain_ab), xlim = c(0, 400), breaks = 30)
Neither of these two community detection algorithms seems to perform notably well. There are other algorithms one could choose, but, in separate analyses, the others performed still worse. It may very well be the topology of text networks is too distinct and presents computational issues.
Likewise, the problem might rest with this network. One could try further cleaning the data in order to produce a hopeful change in topology, but this change seems doubtful because I have already engaged in a fair amount of cleaning. Thus, for now, we will see how topic models perform.
Topic Models
Topic modeling is an unsupervised form of machine learning: we are asking the function/algorithm to provide us with the topics. We provide a number of documents, the number of topics, and perhaps document-level covariates, and we then receive data on which terms fall within each topic and how the topics vary across documents. This description is of course very crude, and I do not intend this post to constitute an introduction to topic modeling. Rather, I want to compare two methods of arriving at the topical structure of a set of documents.
The key question (at least for me, given my research interests) is precisely the number of topics. Just as community detection can be a difficult and convoluted process for network scientists, identifying k is just as difficult (if not more so at this point in time).
The ldatuning package has a FindTopicsNumber()
function that provides a set of diagnostics for models estimated over a set of k values. Let’s use this function to estimate the number of k. We will start with the keyword data (as opposed to the abstract data). Yet, before doing using the FindTopicsNumber()
function, I must create a document term matrix. I begin with akw.art.final
. Numbers, stop words, bioethics, and ethics have been removed from the akw
column. The cast_dtm()
function from the tidytext package makes this task very simple.
article_dtm <- akw.art.final %>%
select(article.id, akw) %>%
count(article.id, akw, sort = TRUE) %>%
cast_dtm(article.id, akw, n)
Now, let’s estimate k. I first load the ldatuning package, and then call the function. It requires the document term matrix, the range of topics to be estimated (for which I use 60-90 based upon the community detection results), the metrics from which the performance will be based (I select all available), the Gibbs sampler methods, a seed number for reproducibility, and the number of cores (I use 10 in order to retain two for my normal computing activities while the function is running).
library(ldatuning)
num_k_ftn <- FindTopicsNumber(
article_dtm,
topics = seq(from = 60, to = 90, by = 1),
metrics = c("Griffiths2004",
"CaoJuan2009",
"Arun2010",
"Deveaud2014"),
method = "Gibbs",
control = list(seed = 1234),
mc.cores = 10L,
verbose = TRUE
)
## fit models... done.
## calculate metrics:
## Griffiths2004... done.
## CaoJuan2009... done.
## Arun2010... done.
## Deveaud2014... done.
Now, I want to plot the results. Arun2010 and Deveaud2014 should be minimized, while Grittiths2004 and CaoJuan2009 should be maximized.
FindTopicsNumber_plot(num_k_ftn)
The results are far from clear, which is normal when estimating k. I am going to choose 64 because it has roughly the best fit among three of the four metrics. In further versions of these analyses, I will replicate analyses on multiple values of k. Note that regardless of the exact value of k, the number of topics best matches the results from the louvain community detection algorithm.
The above models were estimated using keyword data, which is not particularly desirable because there are so few terms per document. I now want to run models with abstract (that is, the abstracts of the bioethics articles) data. First, I will create a dfm object and then use the quanteda package to convert it into an stm object. This latter step is not absolutely necessary, but it makes easier specifying the components of the stm()
function.
# get row == abstract keyword
abstract_words <- data.bioeth.1 %>%
filter(!is.na(abstract)) %>%
unnest_tokens(word, abstract) %>%
anti_join(stop_words,
by = c("word" = "word")) %>%
filter(word != "bioethics" &
word != "ethics") %>%
filter(!str_detect(word, "\\d")) # remove words containing numbers
# get dfm object to give to stm package
abstract_dfm <- abstract_words %>%
count(article.id, word, sort = TRUE) %>%
cast_dfm(article.id, word, n)
# convert to stm object (using quanteda)
library(quanteda)
abstract_stm <- convert(abstract_dfm, to = "stm")
The community detection algorithms provided a starting range for estimating the value of k, but I want to review another option, as well. The stm package has a functionality such that, if you set k equal to zero with spectral initialization, it estimates (a rough starting value for) k, using the algorithm proposed in Lee and Mimno (2014) (see the vignette for further information).
# run stm model to use Lee and Mimno (2014) method
# to estimate k (see vignette p. 13)
searchk_0 <- searchK(documents = abstract_stm$documents,
vocab = abstract_stm$vocab,
K = 0,
init.type = "Spectral",
cores = 10)
summary_searchk_0 <- summary(searchk_0)
The code takes longer to run than I want to take each time I compile this file, so I saved the output and will simply load it before calling the stm()
function. The searchK()
function returned 80 topics as the appropriate starting amount.
stm_0 <- stm(documents = abstract_stm$documents,
vocab = abstract_stm$vocab,
K = 80,
init.type = "Spectral")
## Beginning Spectral Initialization
## Calculating the gram matrix...
## Using only 10000 most frequent terms during initialization...
## Finding anchor words...
## ................................................................................
## Recovering initialization...
## ....................................................................................................
## Initialization complete.
## .....................................................................................................
## Completed E-Step (9 seconds).
## Completed M-Step.
## Completing Iteration 1 (approx. per word bound = -8.120)
## .....................................................................................................
## Completed E-Step (7 seconds).
## Completed M-Step.
## Completing Iteration 2 (approx. per word bound = -7.286, relative change = 1.027e-01)
## .....................................................................................................
## Completed E-Step (7 seconds).
## Completed M-Step.
## Completing Iteration 3 (approx. per word bound = -7.202, relative change = 1.153e-02)
## .....................................................................................................
## Completed E-Step (6 seconds).
## Completed M-Step.
## Completing Iteration 4 (approx. per word bound = -7.180, relative change = 3.053e-03)
## .....................................................................................................
## Completed E-Step (6 seconds).
## Completed M-Step.

## Completed E-Step (6 seconds).
## Completed M-Step.
## Completing Iteration 6 (approx. per word bound = -7.168, relative change = 5.281e-04)
## .....................................................................................................
## Completed E-Step (6 seconds).
## Completed M-Step.
## Completing Iteration 7 (approx. per word bound = -7.166, relative change = 2.334e-04)
## .....................................................................................................
## Completed E-Step (6 seconds).
## Completed M-Step.
## Completing Iteration 8 (approx. per word bound = -7.165, relative change = 9.370e-05)
## .....................................................................................................
## Completed E-Step (6 seconds).
## Completed M-Step.
## Completing Iteration 9 (approx. per word bound = -7.165, relative change = 1.422e-05)
## .....................................................................................................
## Completed E-Step (6 seconds).
## Completed M-Step.
## Model Converged
There are many diagnostics that should be conducted before settling on a final model. However, the purpose of this exercise is to compare community detection and structural topic modeling methods. So, now that our model has converged, let’s check it out. I display in the following block of code the first three topics.
labelTopics(stm_0, c(1, 2, 3))
## Topic 1 Top Words:
## Highest Prob: studies, research, articles, review, literature, published, authors
## FREX: articles, journals, publications, studies, papers, medline, search
## Lift: athletics, coetaneous, coining, cradle, cu, delgado, delgado's
## Score: studies, articles, journals, authorship, publications, papers, medline
## Topic 2 Top Words:
## Highest Prob: consultation, clinical, consultants, community, consultations, care, ethical
## FREX: consultation, consultants, consultations, hecs, competencies, psychiatrists, psychosomatic
## Lift: bxcrrb, hubs, rampart, accredit, accrediting, anticonvulsant, bert
## Score: consultation, consultants, consultations, hecs, competencies, psychiatrists, psychosomatic
## Topic 3 Top Words:
## Highest Prob: paper, based, elements, idea, solidarity, ethical, bioethical
## FREX: introduces, solidarity, elements, idea, ebm, groningen, hospice
## Lift: adduce, admittance, angus, bayertz, biomedicalisation, blasphemous, collage
## Score: solidarity, elements, idea, introduces, ebm, islamic, paper
The four metrics provided for each topic each attempt to capture the relative “importance” of the words (ranked from most to least importance) for each topic. You may refer to the vignette for the stm package to see the equations behind each.
The stm package also has a great plotting function to visualize several aspect of the results. Now, I want to plot the 20 topics that occur with the highest proportion across all documents.
plot.STM(stm_0, type = "summary",
text.cex = .9, # scale text
topics = c(1:20), # choose top 20 topics (by proportion)
n = 4) # list four words for each topic
Conclusion
The purpose of this post has been to review the two key ways to identify the topical structure of a set of texts. The community detection methods have performed rather regrettably, but the structural topic models seem to be better. This outcome is good because the stm package makes it easy to use document-level covariates to predict the prevalence of topics. We could now look, for example, at how topics vary over time or by types of authors (e.g., within different countries or disciplines).
One problem however is that with many topics interpretation can be difficult. Emily Erikson (Yale) has proposed a way to inductively arrive at “themes”. Basically, we use the correlation matrix between topics as a weighted network upon which we run community detection algorithms. This approach gives us a relatively small set of themes. We may then more closely interpret trends within and across topics.
As nice as all of this sounds, I will stop here. Addressing these two issues will be the topic of my next post.
A different interpretation would certainly be the case for the following reason. Communities in the keyword network represent sets of institutionalized positions, what one might consider product categories within the bioethical market. Communities in the abstract network will more likely correspond to the topical or conceptual structure of bioethics discourse.↩


Fishing Guild Member:
Around the Langk Lizardmen Dwellings, if you hunt
Langk Lizardman Sentinels, Langk Lizardman Lieutenants, Langk Lizardman Shaman, or Langk Lizardman Leaders
, you can get the sweet aromatic fluid the water spirits love.
Back


Location
Our Story
Day of the Wedding
Lodging
RSVP
Wedding Party
Home
Location
The wedding will be held on the beach, directly behind the beach house located at:
12458 1st St W
Treasure Island, FL
The reception will be held at this beautiful beach house after the ceremony.


Ambitious, motivated and team-playing computer science and mathematics PhD graduate with sights firmly set upon a dynamic quantitative analyst role. Bringing excellent communication skills, I am a dedicated solution-finder who works well under pressure, either individually or with others to achieve goals.
Developed a large communication website for a client of the police force. Contributed to the company’s core C# library. Learnt a new programming language to resolve an issue within the telephony system of the website.
Implemented fixed income valuation models using Java. Researched calibration of the SABR model. Produced professional standard documentation of models.
Developed MVC web applications using ASP.NET and C#. Implemented generation of automated infrastructure reports. Shadowed experienced quants within the FX team
Implemented systems that improved the efficiency and speed of previous systems. Developed a website in C#/ASP.NET to advertise products
1st Class Honours
Mathematics (A), Further Mathematics (A), Physics (A)


# extends 'base.html' # import 'macro/forms.html' as forms ## import 'macro/utils.html' as utils # set title = 'Posts | Admin' # block content {{super()}}


{{b.title}}
by {{b._user.name}} in the {{b.category}} category
Posted on {{b.createdAt | date: "MMM. dd, yyyy"}}
{{b.snippet}}
Read MoreHere is a collection of my genius (ha!) and tech insights. I'm sure sooner than later I will discuss Formula 1 tech but be assured there will be plenty of SEO and various web development tips/tools/tricks. And lets not forget some general business savviness I've learned over the years.


UI Automation Tools
Action Log
@Author: Ashutosh Mishra, NNG Experiments
@Mindfire Solutions, India
Application is Ready.


pnmsplit(1) pnmsplit(1) NAME pnmsplit - split a multi-image portable anymap into multiple single-image files SYNOPSIS pnmsplit [pnmfile[ output_file_pattern]] DESCRIPTION Reads a Netpbm file as input. Copies each image in the input into a sep‐ arate file, in the same format. pnmfile is the file specification of the input file, or - to indicate Standard Input. The default is Standard Input. output_file_pattern tells how to name the output files. It is the file specification of the output file, except that the first occurence of "%d" in it is replaced by the image sequence number in unpadded ASCII decimal, with the sequence starting at 0. If there is no "%d" in the pattern, pnmsplit fails. The default output file pattern is "image%d". Note that to do the reverse operation (combining multiple single-image PNM files into a multi-image one), there is no special Netpbm program. Just use cat. SEE ALSO pnm(5), cat(1) AUTHOR Written by Bryan Henderson 19 June 2000 pnmsplit(1)


Enter your name below
×
Responsive Chat Template
Cricket Chat Room
Tennis Chat Room
Enter your message
Click to send your message


Sun Bros
Score: 0
Captain: selfish
Challenges solved
| category & points | name | solved at |
|---|
Achievements
| name | description | condition | unlocked at | unlocked by |
|---|
Members
- selfish
- 11amaDrama
- itsallvoodoo
- slashrun
- Gnarsh_Darkwatch
- sparagmos
- mshayda
- gu.ru
- cthulhuismyhero
- moreawesomethanyou


An unique value to identify objects.
More...
#include <ck_id.h>
List of all members.
Detailed Description
An unique value to identify objects.
Member Function Documentation
| bool ckID::operator== |
( |
ckID |
id |
) |
const |
Returns whether this ckID is equal to the right hand side ckID.
- Parameters:
-
| [in] | id | The right hand side ckID. |
- Returns:
- Whether this ckID is equal to the right hand side ckID.
| bool ckID::operator!= |
( |
ckID |
id |
) |
const |
Returns whether this ckID is unequal to the right hand side ckID.
- Parameters:
-
| [in] | id | The right hand side ckID. |
- Returns:
- Whether this ckID is unequal to the right hand side ckID.
| u32 ckID::operator% |
( |
u32 |
n |
) |
const |
Returns the remainder of the value of this ckID devided by the specified value.
This method is used when ckID is used as the key type of ckMap.
- Parameters:
-
| [in] | n | A value to devide the value of ckID. |
- Returns:
- The remainder of the value of this ckID devided by the specified value.
| u32 ckID::getValue |
( |
|
) |
const |
Returns the value of this ckID.
- Returns:
- The value of this ckID.
| static ckID ckID::genID |
( |
|
) |
[static] |
Generates a unique ckID.
- Returns:
- A unique ckID.
| static ckID ckID::genID |
( |
const char * |
str |
) |
[static] |
Generates a ckID whose value is generated from the specified string.
There may be rare occations when the same value is generated from different strings.
- Parameters:
-
| [in] | str | The seed of a unique value. |
- Returns:
- A ckID.
| static ckID ckID::genIDForSystem |
( |
u32 |
value |
) |
[static] |
Generates a ckID from the specified value.
This method is only for system.
- Parameters:
-
| [in] | value | The value of the ckID to be generated. |
- Returns:
- A ckID.
| static void ckID::setCurIDForSystem |
( |
u32 |
value |
) |
[static] |
Sets the value, which is used to generate the next ckID, to the specified value.
This method is only for system.
- Parameters:
-
| [in] | value | A value which is used to generate the next ckID. |
Member Data Documentation
The documentation for this class was generated from the following file:


Methods in this object return OS (Windows or Linux) specific command string.
Requires
- module:JavaType.js
- module:system/System.js
Methods
(static) changeDirectoryAndRunCommand(path, cmd)
Return a command to change a current directory and then run a command.
Parameters:
| Name | Type | Description |
|---|---|---|
path |
String | changing path |
cmd |
String | executing command |
(private, static) changeDirectoryAndRunCommandLinux(path, cmd)
Change directory and run a command with Linux.
Parameters:
| Name | Type | Description |
|---|---|---|
path |
String | path to be removed. |
cmd |
String | executing command |
(private, static) changeDirectoryAndRunCommandWindows(path, cmd)
Change directory and run a command with Windows.
Parameters:
| Name | Type | Description |
|---|---|---|
path |
String | path to be removed. |
cmd |
String | executing command |
(static) getCommandWindow()
Return a command to open command or shell window.
Returns:
"cmd" or "bash" based on the current OS type
(static) normalizePathString(path) → {String}
Return a path string for the current OS.
Parameters:
| Name | Type | Description |
|---|---|---|
path |
String | normalizing path value |
Returns:
normalized path value
- Type
- String
(static) removeDirectory(path)
Return a command to remove a directory.
Parameters:
| Name | Type | Description |
|---|---|---|
path |
String | removing path |
(private, static) removeDirectoryLinux(path)
Remove directory with Linux.
Parameters:
| Name | Type | Description |
|---|---|---|
path |
String | path to be removed. |
(private, static) removeDirectoryWindows(path)
Remove directory with Windows.
Parameters:
| Name | Type | Description |
|---|---|---|
path |
String | path to be removed. |


Description
The sample program creates its own event handler. The handler callback is called when timer ticks occur in the operating system.
Concept
Sample program demonstrates creating your own handler using IHandler.
Task
Notice the time ticks away. These events are intercepted from the operating system and processed in the sample.
Supported
Supported
Supported


--- layout: post title: Implementing Rails APIs like a professional date: 2014-05-08 11:26:24.000000000 -05:00 type: post published: true status: publish categories: - Ruby on Rails tags: [] meta: _publicize_pending: '1' is_reblog: '1' post_id: '3462' blog_id: '9042983' reblog_snapshot: |- O:8:"stdClass":7:{s:5:"title";s:43:"Implementing Rails APIs like a professional";s:4:"type";s:4:"post";s:9:"mime_type";s:0:"";s:6:"format";b:0;s:12:"modified_gmt";s:19:"2014-05-08 09:45:47";s:9:"permalink";s:83:"http://blog.joshsoftware.com/2014/05/08/implementing-rails-apis-like-a-professional";s:7:"content";s:9536:"
generate_api_key
.
[code lang=ruby]
def generate_api_key
loop do
token = SecureRandom.base64.tr('+/=', 'Qrt')
break token unless User.exists?(api_key: token).any?
end
end
[/code]
In this code we use SecureRandom.base64.tr('+/=', 'Qat')
to generate a token. Since, it may contain the special characters like +/=
, we shall replace them with some random alphabets using tr
, so that the API key looks good. Now lets add the before_create
callback.
[code lang=ruby]
before_create do |doc|
doc.api_key = doc.generate_api_key
end
[/code]
Api
in the api
namespace. This helps us avoid the ApplicationController
as APIs do not need the basic controller functionalities like protection from request forgery (the CSRF token), session authentication and the view helpers. Our Api::ApiController
is in fact the ApplicationController
for all our API controllers.
[code lang=bash]
$ rails g controller api/api
[/code]
To ensure we get basic controller functionality we need our Api::ApiController
to inherit directly from ActionController::Base
. So, be sure to replace the code as follows:
[code lang=ruby]
class Api::ApiController < ActionController::Base
# Authentication and other filters implementation.
end
[/code]
Lets take an example to help understand all this better. Lets create an API for events that has two fields name
and occurred_at
that tell us which event occurred at which time. We shall write APIs to get events.
[code lang=text]
$ rails g model event name:string occurred_at:time
$ rails g controller api/v1/events
[/code]
APIs need to be versioned to ensure backward compatibility and future extensibility. The above approach provides us an easy way to do this and keep our code maintainable. Remember to version only when required - v2 would be added when either there is entirely new functionality that would not work with v1 clients, i.e. potentially broken compatibility. (This may be a good time to mention jsonapi.org. This is what the next generation of Web APIs will do, so it's definitely worth a read).
[code lang=ruby]
# app/controllers/api/v1/events_controller.rb
class Api::V1::EventsController < Api::ApiController
end
[/code]
api.example.com
, then we can use options like path
and constraints
[code lang=ruby]
namespace :api, path: '', constraints: {subdomain: 'api'} do
namespace :v1 do
resources :events
end
end
[/code]
Using path
option we can change the namespace name. Now routes look like this
[code lang=ruby]
Prefix Verb URI Pattern Controller#Action
api_v1_events GET /v1/events(.:format) api/v1/events#index {:format=> :json, :subdomain=> "api"}
POST /v1/events(.:format) api/v1/events#create {:format=> :json, :subdomain=> "api"}
new_api_v1_event GET /v1/events/new(.:format) api/v1/events#new {:format=> :json, :subdomain=> "api"}
edit_api_v1_event GET /v1/events/:id/edit(.:format) api/v1/events#edit {:format=> :json, :subdomain=> "api"}
api_v1_event GET /v1/events/:id(.:format) api/v1/events#show {:format=> :json, :subdomain=>"api"}
PATCH /v1/events/:id(.:format) api/v1/events#update {:format=>:json, :subdomain=>"api"}
PUT /v1/events/:id(.:format) api/v1/events#update {:format=> :json, :subdomain=> "api"}
DELETE /v1/events/:id(.:format) api/v1/events#destroy {:format=>:json, :subdomain=> "api"}
[/code]
If we want to process every request as a JSON request by default, then add format: :json
to the defaults
in the namespace options. So any request will now be considered as a JSON request and will give JSON response.
[code lang=ruby]
namespace :api, path: '', constraints: {subdomain: 'api'}, defaults: {format: 'json'}
[/code]
Here is an example of our API controller action.
[code lang=ruby]
class Api::V1::EventsController < Api::ApiController
respond_to :json
def index
@events = Event.all
respond_with @events
end
end
[/code]
Api::ApiController
itself so that it is applicable for all the API controllers. There are different ways to authenticate:
authenticate_or_request_with_http_token
authenticate_or_request_with_http_token
(ref: token auth api), our code looks like this.
[code lang=ruby]
class Api::ApiController < ActionController::Base
private
def authenticate
authenticate_or_request_with_http_token do |token, options|
@user = User.where(api_key: token).first
end
end
end
[/code]
Now, in our EventsController
we need to ensure we add the filter. (Note: Rails3 would require this to be before_filter
).
[code lang=ruby]
class Api::V1::EventsController < Api::ApiController
respond_to :json
before_action :authenticate
def index
@events = Event.all
respond_with @events
end
end
[/code]
Now, to test this API out we can use curl.
[code lang=text]
$ curl -H "Authorization: Token token=WwEsPpOCYMsyPsmmvKBqQDOaEJ4t" http://localhost:3000/events
[/code]
If we pass the api_key
in the request header (i.e the X-API-KEY
header field), then we need to do the authentication manually.
[code lang=ruby]
def authenticate
api_key = request.headers['X-Api-Key']
@user = User.where(api_key: api_key).first if api_key
unless @user
head status: :unauthorized
return false
end
end
[/code]
Again, we can test this with curl.
[code lang=text]
$ curl -H "X-Api-Key: WwEsPpOCYMsyPsmmvKBqQDOaEJ4t" http://localhost:3000/v1/events.json
[/code]
NOTE 1 Never pass the API key in the params(for example: http://test.com?api_key= WwEsPpOCYMsyPsmmvKBqQDOaEJ4t
or in the POST or PUT parameters). The params will get logged and the API keys would be seen in the logs. If someone has access to your logs or purges the logs or even by mistake pastes a log snippet on stackoverflow, then God help you! We can of course use filtering configuration in our application but the best way is to avoid this situation and pass the API key in the request header always.
NOTE 2 Always use the HTTP status codes for different response status to make the HTTP status code more readable.
[code lang=ruby]
render status: :unauthorized # 401
render status: :too_many_requests # 424
render status: :unprocessable_entity # 422
# and many more!
[/code]
In my next post, I shall talk about Request throttling. The information here is enough for you to get started on writing good Rails APIs that are versioned, clean and authenticated. If you want to see a sample working code, see Github.
Feedback is welcome.


  And (it is forbidden unto you) that ye should have two sisters together, except what hath already happened (of that nature) in the past. Lo! Allah is ever Forgiving, Merciful. |
| 004:023 Rashad | : |
Prohibited for you (in marriage) are your mothers, your daughters, your sisters, the sisters of your fathers, the sisters of your mothers, the daughters of your brother, the daughters of your sister, your nursing mothers, the girls who nursed from the same woman as you, the mothers of your wives, the daughters of your wives with whom you have consummated the marriage - if the marriage has not been consummated, you may marry the daughter. Also prohibited for you are the women who were married to your genetic sons. Also, you shall not be married to two sisters at the same time - but do not break up existing marriages.  It would not be a sin to marry her if you did not have carnal relations with her mother. You are forbidden to marry the wives of your own sons and to marry two sisters at the same time without any adverse affect to the such relations of the past.  You can also visit Qurany Concepts for browsing the Quran themes or topics. If you want to search the Holy Quran for keywords, please visit Qurany Keywords.


All site visitors can set the way albums are displayed in the Gallery module. Note: This field may be disabled.
Gallery displayed in List View
Gallery displayed in Card View
Gallery displayed in Standard View
Related Topics:


# extends 'base.html' # import 'macro/forms.html' as forms ## import 'macro/utils.html' as utils set title = 'Ads | Admin' # block content


Ali (Per Graffman), a cabdriver living in Sweden, thought he had left his guerrilla past behind him 18 years ago, but it catches up with him in the opening scenes of "Before the Storm". Addressing him as "Captain," a woman fare (Nasrin Pakkho) knows not only who he is, but what he is capable of doing. Unless he agrees to assassinate the Swedish manufacturer who has been selling trucks that are converted into rocket launchers against their people, her compatriots (and his formerly) will kill the wife and son that he left behind. Although the film does not identify their nationality, it is implicit that they are Kurds since they are being persecuted for their language.
Ali has a new family in Sweden. Now married to a Swedish woman, they have two teenaged daughters. The younger has a classmate named Leo (Emil Odepark) who has a crush on her, but whose affections are not returned.  But none are as cruel as Danne (Martin Wallström), an older boy who makes Leo's life a living hell. At knifepoint, he forces Leo to march naked into the girl's locker-room to don the panties of Ali's daughter, the girl he loves.
As with students in the American news, Leo decides to take matters into his own hands. He steals his mother's 9 millimeter pistol (she is a police officer) and lures Danne into a wooded area where he forces him to get down on his knees and apologize. In the ensuing altercation, Leo shoots Danne in the chest.
The remainder of the film focuses on the crossed paths of Ali and Leo. One is fleeing from the law, particularly from his mother who has discovered a missing bullet in her pistol's chamber. The other is fleeing from his duty as assassin, coerced as it is. In a riveting scene, the two sit in Ali's kitchen in the middle of the night. Just moments after warning Leo that he must turn himself in (he can not run forever), Ali receives a phone call from his blackmailers. They put his first wife on the line to plead for her life. Overhearing this chilling conversation, Leo discovers that Ali is also in the hands of fate. Both of these characters, longing for normalcy and love, are wrenched into another, more frightful reality by circumstances beyond their control. In one case, war in the Mideast. In the other, a sadistic high school bully.
By making his two lead characters murderers, young first-time director Reza Persa (born in Iran but living in Sweden since 1980) is confronted with the challenge of making us empathize with decent characters doing indecent things. As he told the audience after the showing, there is no greater challenge facing an artist. Although some Arabs have criticized him for stereotyping them as terrorists, he explained that most Swedes appreciate the film for showing the powerful forces that can make people take desperate actions. In addition, the film effectively confronts stereotypes about the Swedes, although of the opposite sort. When the film debuted, critics found it hard to believe that a Swedish schoolchild would kill a bully in this fashion, but only three months ago such an event took place for the first time in this bland, social-democratic country. Showing far greater gullibility, some critics also questioned whether Swedish manufacturers could be war profiteers. Evidently they never heard of Alfred Nobel, who funded his annual prizes with money he made from the sale of dynamite, a product he invented.
Louis Proyect Marxism mailing list: http://www.marxmail.org
The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


TRUE CRIME ***1/2 (out of five stars) A review by Jamey Hughton
Starring-Clint Eastwood, Isaiah Washington, Denis Leary and James Woods Director-Clint Eastwood Rated-14A Released March 1999 Warner Bros.
MOVIE VIEWS by Jamey Hughton: http://welcome.to/movieviews
Veteran actor Clint Eastwood has never looked as grizzled as he does in True Crime, his latest directorial effort. When Steve Everett (his newest character) gets angry at someone, he glares them down with those famous Dirty Harry eyes, furrows his brow and frowns like a grizzly bear who's just lost his cubs. Eastwood has played some particularly despicable characters in his time, but Everett could just take the cake.
He gets my vote, at least, partly because `Ev' is a drunken affair-a-week womanizer who has many relationship problems, very few of which are with his wife (Diane Venora). When his colleague at the Oakland Tribune is in an ugly car wreck and dies, Everett must take over for her at a vital interview session. The interview is with Frank Beacham (Isaiah Washington), a death-row inmate set to die at midnight for the murder of a pregnant convenience store clerk. Eastwood furrows his brow.
As Everett gradually finds information, he realizes that Beacham could very well be innocent. He interviews a key witness (Michael Jeter), who claims that he burst in the door at Pocum's Foods because his car had overheated, only to see Beacham standing over the dead woman's body, blood on his suspenders, gun in hand. But Everett protests: how could he have seen the gun, which was lowered by his side, with the potato chip rack in front of him? Jeter doesn't know what he's talking about. Eastwood furrows his brow.
Crinkled expressions and all, Clint is the centre of energy of True Crime. The film is by no means a standard action/suspense yarn, but a thoughtful human story in which the characters come before the shoot-outs.  But the best scenes are ones that feature Eastwood duking it out with those in authority over him. Denis Leary, as Everett's editor and boss, has more than a few memorable moments of restrained anger (you see, Ev is sleeping with his wife). But hands down, the most enjoyable segments of the film are when James Woods is on camera. Playing the big boss Alan Mann, Woods and Eastwood create amusing chemistry and laugh-out-loud punchlines.
When True Crime opts for a high-speed chase to the Governer's house at the finale, the quality of film-making takes an abrupt nosedive. Eastwood was so successful with colorful character portraits that he didn't need to switch lanes. True Crime is a tension-building, intriguing drama showcase for the talented director and star. This is a road block he could have easily dismissed (I furrow my brow). ...........................
MOVIE VIEWS by Jamey Hughton: http://welcome.to/movieviews
The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews