I have mentioned the preserve
and restore
commands in both NC-151 and NC-152 before, but now I
want to emphasize their use to you.
Most Stata commands involve restricting operations to the relevant subsample of the data. The relevant subsample may be explicitly specified by the user,
. regress mpg weight if foreign
and may also include implied restrictions:
. regress mpg weight if foreign
----------
|
+--> meaning mpg<. and weight<.
We obtain the explicit restrictions by parsing
...
syntax ... [if] [in]
...
...
and the easiest way to deal with the explicit restrictions thereafter is to create a marker variable containing 1 if we are to use the observation and 0 otherwise:
tempvar touse
mark `touse' `if' `in'
Then, later in our code, we just code if `touse'
at
the end of any data-processing commands.
I do not want mark
to be a mystery to you. mark `touse' `if' `in'
has the same effect as if you coded
gen byte `touse' = 0
replace `touse' = 1 `if' `in'
The only difference is that mark `touse' `if' `in'
executes more
quickly.
To handle the implied restrictions, we reset `touse'
to contain 0 in the appropriate places. If we were writing regress
and wanted to exclude observations with variables
with missing values, we could loop across the variables in `varlist'
and reset `touse'
to
contain 0 wherever any of the variables contained missing values. We could
write the following code:
foreach i of local varlist {
replace `touse' = 0 if `i'==.
}
Coding
markout `touse' `varlist'
has the same effect, but is faster, both to code and to execute. Thus if you
want to honor if
and in
and
if you want to exclude all observations of the variables specified with missing
values, you can code
...syntax ... [if] [in]
...tempvar touse
mark `touse' `if' `in'
markout `touse' `varlist'
...
or, you can code something even shorter:
...syntax ... [if] [in]
...marksample touse
...
marksample touse
is equivalent to tempvar touse
followed by mark `touse'
`if' `in'
followed by markout `touse'
`varlist'
.
mark
, markout
, and marksample
are sufficient for most cases, especially once
you realize that you can use markout
repeatedly. For
instance, say that you want to exclude observations with missing values, honor
the if
and in
, and that there
is one other variable, say, id, that also must not be missing. You could code
syntax
...[if] [in]
...marksample touse
markout `touse' id
That you can use markout
repeatedly should not
surprise you because markout
really does nothing more
than replace
a mark variable with 0 when variables in
varlist contain missing values.
mark
and markout
may be
sufficient for your purposes, but in some cases, you may need to imply other
restrictions as well or instead. There is nothing stopping you from
programming
replace `touse' = 0 if
...
Anyway, by the time you complete all of this, you will have a variable `touse'
that contains 1 if you are to use the observation
and 0 otherwise. In summary, the outline I recommend is
...syntax
...[if] [in]
...marksample touse
markout `touse'
... /* if that is relevant */replace `touse' = 0 if
... /* if that is relevant */ ...
Sometimes I do not bother to do this; the code is short enough that I can see
that I am restricting to the relevant sample by just tacking `if' `in'
onto the end of the relevant commands. That is
fine, but be careful. If you were writing regress
,
there might come a point in your code where you would need to obtain the mean
of the dependent variable, and you might be tempted to code
summarize `depvar' `if' `in'
local meandep = r(mean)
That would not be right. Later in the code, the regression will be estimated
over the subsample of data for which none of the independent variables is
missing, and the mean you just calculated did not impose that restriction. The
only safe way to proceed in this more complicated code is to include the code
to create `touse'
,
marksample touse
and then
summarize `depvar' if `touse'
local meandep = r(mean)
Tacking if `touse'
onto the end of commands does not
seem difficult, but there are programs in which it is, such as those that need
to sort the data due to a by()
option.
Say that in some piece of code you are writing, you need to obtain the mean of
`depvar'
by the `by'
variables. You might code
tempvar mean
sort `by'
by `by': gen `mean' = sum(`depvar')/sum(`depvar'<.) if `touse'
by `by': replace `mean' = `mean'[_N] if `touse'
That will not work. The gen `mean'
statement is okay,
but it may not produce a mean in the last (_N) observation because the _Nth
observation might have `touse'==0
(meaning it is
excluded). The right way to code this is
tempvar mean
sort `touse' `by'
by `touse' `by': gen `mean'=sum(`depvar')/sum(`depvar'<.) if `touse'
by `touse' `by': replace `mean' = `mean'[_N] if `touse'
That is, rather than forming by groups, I form two sets of by groups, one for
`touse'==0
and another for `touse'==1
, and then I just work on the
`touse'==1
group.
In extremely complicated programs, restricting the calculation to the relevant subsample can get even more difficult. When it becomes too complicated for you (and it quickly becomes too complicated for me), there is a trick that can help:
preserve
quietly keep if `touse'
Use those two commands, and afterwards, you can forget about restricting to the relevant subsample because the only data in memory make up the relevant subsample. When your command ends—no matter how—Stata will automatically restore the original data just as if the data were never disturbed from memory.
preserve
is a great solution, but it takes time to
execute because it literally makes a copy of the dataset on disk, and later,
restores the data literally reading the copy back into memory. Do not let this
frighten you away from using preserve
—Stata is
faster at disk operations than most software—but use preserve
only when you really need it.
There is another command, restore
, for use along with
preserve
, but it is used only rarely. When you code
restore
the previously preserved data are reloaded, and the preserve
is canceled, which is to say, the disk file
containing the copy of the original data are erased.
It is not your responsibility to code restore
; the
restoration is automatic, but if you want the entire dataset back early because
of something you are doing, you can invoke restore
for
yourself.
A variation on restore
is
restore, preserve
restore, preserve
does the same thing as restore
, but it does not cancel the automatic restore when
your program ends; the disk file containing the copy of the original data are
not erased yet. restore, preserve
has the same effect
as coding restore
followed by preserve
,
restore
preserve
but it is faster. If you gave the two commands in sequence, the data would be
reloaded and the disk file erased, and then the data would be copied to a new
disk file. The single restore, preserve
, on the other
hand, reloads the data and stops right there, leaving the existing disk file
behind.
A final version of restore
is
restore, not
which does nothing but cancel the automatic restoration. Imagine that you are
writing a program that drastically rearranges the data in memory—think of
collapse
and reshape
as
examples. At some point, the data will be arranged in an in-between state that
is not at all like the original nor the final. The way to code such programs
is
program
... ...preserve
...restore, not
end
The following are equivalent:
syntax [varlist] [if] [in]
...marksample touse
and
syntax [varlist] [if] [in]
...tempvar touse
mark `touse' `if' `in'
markout `varlist'
It does not matter which you code.
I prefer marksample
because I am less likely to make a
mistake by forgetting something. For example, pretend that I coded my mark
command as
mark `touse' `if' `ip'
I meant to type `in'
at the end, but I typed `ip'
instead.
`ip'
is an error, but is not an error that Stata will flag because
`ip'
will just evaluate to nothing. Result: my program ignores
in
range when the user specifies it.
marksample
is smart in that it accesses the
information left behind by syntax
, and marksample
never forgets anything or makes a mistake. Less
typing means less chance of error, so I prefer marksample
:
syntax [varlist] [if] [in]
...marksample touse
That is, I prefer marksample
when marksample
does what I want it to do. marksample
applies all the usual restriction rules, but I
have written the rare program where I do not want all the usual rules applied.
Pretend that I was writing a program to report on the pattern of missing values
in varlist. If I coded as above, and my program otherwise worked, my program
would report that none of the variables have missing values, and it would
report that even when it was not true. Why? Because marksample
means
syntax [varlist] [if] [in]
...tempvar touse
mark `touse' `if' `in'
markout `varlist'
with the emphasis on markout
. marksample
automatically marks out the missing values in
varlist. In this pattern-of-missing-values command, I do not want to mark out
the missing values because it is precisely their pattern my program intends to
examine. I could code
syntax [varlist] [if] [in]
...marksample touse, novarlist
because marksample
has an option to omit doing the
markout
, but I would probably code
syntax [varlist] [if] [in]
...tempvar touse
mark `touse' `if' `in'
It is just a matter of style. Both work equally well, and there is no difference between their outcomes.
Basically, I use marksample
when I code a new command
following "standard" rules and use mark
and markout
when my program is unusual. Thus, when I see mark
in my programs, I know something is up, and if I am
about to modify the code, I had better look twice to make sure I understand
what is going on.
Suppose that you want to write a program that reports something, and you want
to allow the user to to be able to use your program with Stata's by varlist:
prefix, so that the user could type
. myprogram mpg weight
or
. by foreign: myprogram mpg weight
This is easy to program and, in fact, is related to restricting your command to the relevant sample, which we have been discussing. Remember, if the user types the second command, the user wants to see the same result as if he or she had typed
. myprogram mpg weight if foreign==0
. myprogram mpg weight if foreign==1
assuming that variable foreign takes on the values 0 and 1. by
amounts to repeatedly performing myprogram and each time
restricting myprogram to the appropriate sample.
Half the problem we have already discussed: the solution is to use mark
or marksample
.
The other half has to do with repeatedly performing the command, and Stata does
that automatically. The bottom line is that if you code your program so that
it restricts itself to the relevant sample using mark
or marksample
, all you have to do to make your program
work with the by prefix is change
program myprogram
...end
to read
program myprogram, byable(recall)
... ^^^^^^^^^^^^^^end
In Stata jargon, a program that allows the by varlist:
prefix is said to be byable. If you make
the above change, and the user types
. by foreign: myprogram
...
your program will be executed once for each by group, and mark
(or marksample
) will
automatically know to set the `touse'
variable to 0
for observations outside the by group on each iteration. From the point of
view of your program, results really are as if the user typed
. myprogram mpg weight if foreign==0
. myprogram mpg weight if foreign==1
except that `if'
is not what is defined; rather, the restriction is
passed through a side door to mark
and marksample
.
There are two ways to write byable programs, called byable(recall)
and byable(onecall)
.
byable(recall)
is easiest to program and is
wonderfully robust, and you should use it whenever you can.
Using byable(recall)
when executing your program
multiple times would not produce the desired result. This can happen for
substantive or programming reasons. Substantively, perhaps you want by varlist:
to mean something
different. For example, perhaps
. by foreign: my_other_command mpg weight
is not meant to run my_other_command
twice, once for
foreign==0 and again for foreign==1, but to report estimates that are adjusted
for stratification on foreign. That would be an example where the by(onecall)
would be required, but I do not much like the
intended misuse of Stata syntax. Better would be
. my_other_command mpg weight, by(foreign)
or
. my_other_command mpg weight, strata(foreign)
Users expect by
on the left to mean "repeat,
independently", and if that is not what you mean, then by
belongs someplace else. byable(recall)
can be inappropriate for programming reasons.
Pretend that you are writing a command to create a new variable:
. mygenerate newvar =
...
Within your program, obviously, you will have a Stata generate
command to create the new variable newvar. Now you
want to generalize your program to allow
. by foreign: mygenerate newvar =
...
Think about simply modifying your program to be byable(recall)
. On the first call, your program would
execute, restricting itself to the appropriate sample, and generate
newvar; results would be perfect. On the second
call, your program would execute again, restricting itself to the
now-appropriate sample, and then your program would generate
newvar and stop. newvar already exists. We
generated that variable on the first call. Your program does not need generate
; it needs replace
.
Typically, in such programs, other differences will arise, too.
byable(onecall)
is for handling those kinds of
problems. In byable(onecall)
programs, your program
is called just once, but various macros are filled in to let you know that the
user specified by
. It is your program's
responsibility to do the appropriate thing with the information you are passed,
which includes
`_byindex'
the name of temporary variables containing
1, 2, ... that denotes the individual by groups.
You are passed other information, too, but you can generally ignore it. `_byindex'
is all you need. It is your responsibility to
write your program to exploit this extra piece of information, but that is
usually not difficult. In byable(onecall)
programs,
mark
and marksample
work no
differently from how they do in non-byable programs; your program is called
once, and you put together the `touse'
information
with `_byindex'
and use it appropriately.
Only once did I find it necessary to write a byable(onecall)
program, and that was egen
. I do not expect ever to have to write a byable(onecall)
program again, but one never knows. On the
other hand, I toss in a byable(recall)
at the top of
most of my programs. (I admit that I have written more than one byable(onecall)
program, but only once did I really have no
alternative. The other times, I was translating an old program written before
byable()
was available. The program had a by()
option that handled by processing, and I found it
convenient to use byable(onecall)
so that I could
dummy up a by()
option and just let the old code work.
That was convenient, but if I were writing the programs from scratch, I would
have written them as byable(recall)
programs.
As mentioned at the beginning of this section, by
is a
prefix command. To see a list of all Stata prefix commands, type help prefix
. If you want to create a command that works
with one of Stata's prefix commands, see help program
properties
to learn how. Also, if you want to write your own prefix
command, there are some undocumented subroutines that can help you. Type help _prefix
in your Stata Command window for more
information on these subroutines.
I want to discuss lists and their manipulation. A list is nothing more than a sequence of tokens (words) separated by spaces
1.2 2.5 3.3 myfile.dta thisfile.dta thatfile.dta mpg price weight "element 1" "element 2" "third element and last" `"element 1"' `"element 2"' `"third, "quoted" element"'
Each of the lists above contain three elements; real lists may contain any number of elements, including zero. White space separates one element from the next, and the number of blanks used does not matter. Quotes—either simple or compound—may be used to bind elements that themselves contain blanks or other quotes. The style of quoting—and whether quotes are even used—can be mixed within a single list:
myfile.dta "element 2" `"third, "quoted" element"'
The most common type of list in Stata is the varlist—a list of variables—but lists of anything are possible.
List manipulation refers to
The most common way lists are created in Stata is by the simple act of parsing:
syntax varlist
...
The local macro `varlist'
now contains a list of variable names.
I sometimes use another construction for creating lists. Right now, I am going to show you an artificial example, but shortly, I will show a more reasonable one.
I want to create a list containing the numbers "1 2 3 ... `n'
". The code fragment is
...local mylist
forvalues i =1/`n' {
local mylist "`mylist' `i'"
}
...
The local mylist
statement clears local macro mylist
—it is
equivalent to typing
local mylist ""
or
local mylist = ""
The forvalues
loop simply generates the `n'
list elements. The
line that reads
local mylist "`mylist' `i'"
appends the next element to the end of mylist. Trace the execution. The first time this statement is executed, the line reads
local mylist " 1"
and `mylist'
now contains " 1"
. The list
does not need the leading blank, which is just an artifact of the code. More
complicated code could eliminate it, but the leading blank is of no importance
one way or the other, so I opt for simpler code.
The second time local mylist "`mylist' `i'"
is
executed, `mylist'
contains " 1"
and
`i'
contains "2"
, so the expanded line
reads
local mylist " 1 2"
and `mylist'
is reset to contain " 1 2"
, and so on.
As a more realistic example, I once had a programming problem where I needed to
create temporary Stata datasets containing the data corresponding to each group
of a by()
option. I began by processing the by()
option in the usual way.
syntax ... [, BY(varlist) ... ]
marksample touse
tempvar group
egen `group' = group(`by') if `touse'
In this code, I used the egen
function rather than the
mark-sort-and-sum logic of lecture 3. That's not important—one is the
equivalent of the other—either creates a `group'
variable that is encoded 1, 2, ..., k. I did need
to know how many groups I had:
quietly summarize `group' if `touse'
local k = r(max)
The program I was about to write was long and involved, and I simply did not
want to be bothered with making sure I said if `touse'
at the end of all the relevant commands. Second, there was going to be lots of
sorting, merging, and other manipulations, so keeping track of which
observations were relevant was too difficult for me. So, my next two lines
read
preserve
quietly keep if `touse'
Now, remember my problem: I needed to create temporary Stata datasets
containing the data corresponding to each group of the by()
option. That is, I needed one dataset for `group'
==1, another for `group'
==2,
and so on. (What I intended to do with those datasets is not important.) Here
is what I did:
quietly {
forvalues i = 1/`k' {
restore, preserve
/* reload the preserved data */keep if `group'==`i' & `touse'
tempfile work
save `"`work'"'
local files `"`files' `"`work'"'"'
}
}
That is, I looped across the groups, reloaded the original data, kept the
subsample, got a temporary filename, saved the data, and added the temporary
filename to a list called `files'
. Also note that I
did not bother to initialize `files'
—there is no
local files
line. I could have included this before
the forvalues
, and perhaps I should have for clarity,
but since `files'
had never been used previously in my
program, `files'
was by definition empty.
I hope all the quotes are not confusing, but just in case they are, in my first draft of the above code, rather than coding
save `"`work'"'
I coded
save `work'
and rather than coding
local files `"`files' `"`work'"'"'
I coded
local files "`files' `work'"
That's more readable but I realized that on some people's computers, the
temporary filename `work'
might have a blank in it
because their computers store temporary files in the directory
c:\temporary files\
or some such place. I was attempting to
create the list
file_for_group_1 file_for_group_2 ...
but if the filenames for the groups had blanks, I would need
"file_for_group_1 " "file_for_group_2 ...
Then I realized that some strange person out there might have a temporary directory with quotes in its name:
`"file_for_group_1 "' `"file_for_group_2 "' ...
would be even safer. Hence,
save `work'
became
save "`work'"
became
save `"`work'"'
and
local files "`files' `work'"
became
local files `"`files' "`work'""'
became
local files `"`files' `"`work'"'"'
Just understand that a list is `"thing1 thing2 thing3 ..."'
. If the
things are not quoted, that's fine, but then the things are distinguished by the
blanks that separate them. If you want to allow for the things themselves to
contain blanks, you can code the list `""thing1" "thing2" "thing3""'
. If
you want to allow for the things themselves to also contain quotes, you can code
the list `"`"thing1"' `"thing2"' `"thing3"'"'
. This last is safest but
is the most difficult to read.
Anyway, returning to my problem, I had just formed a list of datasets. Later in
my code, when I needed to reaccess the datasets, I could obtain the names from
my list in `files'
, which brings us to the second issue on dealing with
lists.
There are several ways to step through list elements. I will show you two of them, one of which you have seen before.
foreach i of local list {
... /* logic referring to `i' */}
So, in my separate-datasets-per-by-group example, when the time came to reaccess the datasets, I could code
foreach i of local files {
use `"`i'"', clear
...}
There is another way to step through lists:
local i = 1
local token : word `i' of `list'
while `"`token'"' != "" {
... /* logic in terms of token */local i = `i' + 1
local token : word `i' of `list'
}
This method hinges on the local ... : word ... of ...
command. It's
pretty readable:
word 1 of alpha beta gamma
is alpha"
word 2 of alpha beta gamma
is "beta"
word 3 of alpha beta gamma
is "gamma"
word 4 of alpha beta gamma
is ""
Try it interactively:
. local mylist "alpha beta gamma"
. local x : word 2 of `mylist'
. display "`x'"
You will discover that `x'
contains beta
. Quotes work exactly as
you would expect:
. local mylist `"alpha `"beta or b"' gamma"'
. local x : word 2 of `mylist'
. display "`x'"
`x'
will contain beta or b
.
The first method is now the preferred method, as it requires less programming and its logic is transparent.
Sometimes, you have a list and need to delete an element from it, (say, the
`i'
th one). The trick to doing this is in understanding how `*'
works. The formal definition of `*'
is
`*'
is the tokens `1', `2', `3'
, ..., appended one after the
other, with a single blank in between.
For example, if the macros contain
| `1' | `2' | `3' | `4' | `5' | `*' contains |
|---|---|---|---|---|---|
| mpg | weight | turn | rep78 | nothing | mpg weight turn rep78 |
| mpg | weight | nothing | — | — | mpg weight |
| mpg | nothing | — | — | — | mpg |
| nothing | — | — | — | — | nothing |
The location of the first nothing determines where
`*'
ends. The table above could just as well be
presented as
| `1' | `2' | `3' | `4' | `5' | `*' contains |
|---|---|---|---|---|---|
| mpg | weight | turn | rep78 | nothing | mpg weight turn rep78 |
| mpg | weight | nothing | rep78 | nothing | mpg weight |
| mpg | nothing | turn | rep78 | nothing | mpg |
| nothing | weight | turn | rep78 | nothing | nothing |
Consider the second line. Since `3'
contains nothing, it simply does not matter that `4'
contains rep78
; `*'
contains only "`1' `2'
",
meaning mpg weight
.
Now, let me clarify the meaning of nothing—it is nothing, "", or nullstring. They all mean that the macro has no contents.
---------------------------------------------------------------------- If`1'
contains "mpg"`2'
contains "weight"`3'
contains "turn"`4'
contains "rep78" and I typelocal 3 ""
, clearing macro`3'
,`*'
contains "mpg weight" ----------------------------------------------------------------------
nothing is quite distinct from blank:
---------------------------------------------------------------------- If`1'
contains "mpg"`2'
contains "weight"`3'
contains "turn"`4'
contains "rep78" and I typelocal 3 " "
,`*'
contains "mpg weight rep78" ----------------------------------------------------------------------
local 3 " "
is the key to deleting elements from a list. To delete the
third element from list `list'
, tokenize
it, blank out the
element, and resave the list:
tokenize `"`list'"'
local 3 " "
local list `"`*'"'
To delete the `i'
th element from the list—where `i'
is a
local macro containing an integer—
tokenize `"`list'"'
local `i' " "
local list `"`*'"'
We have already seen how to add an element to the tail of a list:
local list `"`list' "'
You can also add elements to the middle of a list, but in my experience, that is seldom needed. Adding in the middle is a variation of deleting elements. To add newelement between the third and fourth elements of a list:
tokenize `"`list'"'
local 3 `"`3' newelement"'
local list `"`*'"'
To add newelement between elements `i'
and `i'
+1, where
`i'
is a macro containing an integer,
tokenize `"`list'"'
local `i' `"``i'' newelement"'
local list `"`*'"'
Related to lists but serving a different purpose are vectors of macros. There have been times when I have wanted to create macros
`x[i]', i= 1, 2,
...
Stata does not support the `x[i]'
notation. The right
notation is
`x`i'', `i' = 1, 2,
...
Do you see how this works? Be very mechanical and expand `x`i''
,
assuming a previous local i 1
statement.
`x`i''
when`i'
contains 1 expands to`x1'
because you expand the inside first
Similarly,
`x`i''
when`i'
contains 2 expands to`x2'
There, then, is my vector. If I want to set the `i'
th element, I type
local x`i'
...
When I want to use the `i'
th element, I type
... `x`i''
...
Vectors and lists are obviously related; vectors and lists differ in that
For some problems, lists are preferable; for others, vectors work best; and for yet others, it does not make much difference.
Remember my separate-datasets-per-by-group example, which I coded using a list:
preserve
quietly keep if `touse'
quietly {
forvalues i = 1/`k' {
restore, preserve
keep if `group'==`i' & `touse'
tempfile work
save `"`work'"'
local files `"`files' `"`work'"'"'
}
...
foreach i of local files {
use `"`i'"', clear
...
}
...
}
I could just as well have done this using macro vectors:
preserve
quietly keep if `touse'
quietly {
forvalues i = 1/`k' {
restore, preserve
keep if `group'==`i' & `touse'
tempfile file`i' lt;--
save `"`file`i''"' lt;--
}
...
forvalues i = 1/`k' { lt;--
use `"`file`i''"', clear lt;--
...
}
...
}
This code is better than the previous code, but I really did use the previous code in the example I refer to. That is, I tend to overuse lists, and there is nothing too wrong with that.
 Imagine that
`k'
was a very large number (`k'
, you will remember, is the number
of groups in the data). In my previous list-based solution, I would attempt to
stack all of those filenames into a single macro. The filenames that
tempfile
produces are roughly 18 to 28 characters long, depending on your
operating system.
The maximum length of a macro is 165,200 characters in Stata/IC and 1,081,511 in Stata/SE and Stata/MP. Now, imagine what would happen as the number of files grew large. The vector-based program would continue to work, but the list-based program would fail once the number of characters we were trying to store in our macro exceeded the limit. (In the real problem underlying this example, there were only 100 groups, so my inferior program worked.)
Turning to the new code, note the lines that read
tempfile file`i'
save `"`file`i''"'
There are no mistakes. file`i'
is the name of the `i'
th file
macro, and `file`i''
is its contents. Pretend that `i'
contains
5. Then, the lines read
tempfile file5
save `"`file5'"'
This also emphasizes that when you form a vector of macros in this way, you are producing the result by trickery with the name, which means you had better be sure the trickery works out.
The same trickery can also be used to produce multidimensional arrays of macros, but I have never yet found a use for it. If you wanted to do this, I suggest
| x`i'_`j' | to define the [`i',`j' ] macro |
| `x`i'_`j'' | to refer to its contents |
The underscore in the middle is important, although you could use whatever
character you liked. Pretend that you left the underscore out, creating macros
named x`i'`j'
(and accessing them as `x`i'`j''
). Now, consider
the macro with the resulting name x123. Does it refer to x`i'`j'
for
`i'
=1 and `j'
=23 or to the macro for `i'
=12 and
`j'
=3? The answer is that both [1,23] and [12,3] produce the 123 suffix,
and therein lies the problem. The intervening underscore solves that problem.
As I said, not once have I needed a two-dimensional macro array.
You can also create global macro vectors—the logic is the same, but the way you nest global macro expansions is different from local expansions. It takes two characters to expand a local macro,
`...'
the open and the close quote. Thus the meaning of something complicated such
as `x`i''
is clear because the open and close quotes match:
+----+
| |
`x`i''
| |
+-+
The global macro expansion character is a single dollar sign. Thus consider
$x$i
That is not a vector; that is a request to expand the global macro $x
followed by the expansion of $i
. The way you write the nested expansion
is
${x$i}
That is, Stata allows the $
to be followed by an open brace and, if it
is, finds the matching close brace. ${x$i}
makes it clear that the
inside, x$i
, must be expanded first. If $i
contains 1, x$i
expands to x1
, so ${x$i}
expands to ${x1}
, which refers to
the contents of the global macro $x1
.
You can mix global and local macro expansions.
${x`i'}
refers to $x2
if local macro `i'
contains 2.
Consider parsing the following command:
. mycmd 3 earnings age if sex=="female", follow
After invoking our program, we have the following at our disposal:
| `0' |
|---|
| 3 earnings age if sex=="female", follow |
| `1' | `2' | `3' | `4' | `5' | `6' | `7' | `8' |
|---|---|---|---|---|---|---|---|
| 3 | earnings | age | if | sex== | female | , | follow |
None of the parsing tools we have so far learned can handle this problem.
syntax
parses `0'
, but the command does not follow standard Stata
syntax because of the leading number.
The other approach we learned—using the positional parameters—will
not help because the quotes around "female"
have been lost. Looking at that, we
cannot tell whether the user typed if sex==female
(referring to two
variables) or if sex=="female"
. In fact, we cannot even tell whether the
user typed if "sex==" "female"
.
The solution to parsing nonstandard syntax is to use gettoken
and, in
fact, we can parse mycmd
by coding
gettoken number 0 : 0
syntax varlist [if] [in] [, Follow]
The syntax of gettoken
is
gettoken newmac : oldmac [, options ]
and
gettoken newmac oldmac : oldmac [, options ]
where oldmac and newmac are macro names. gettoken
looks at oldmac, obtains the next syntactic element (token) from
it, and stores it in newmac. In the second syntax, gettoken
also takes what is left over and uses that to replace oldmac. In
the first syntax, the contents of oldmac remain unchanged.
Consider `0'
containing
| `0' |
|---|
| 3 earnings age if sex=="female", follow |
and coding
gettoken number : 0
or
gettoken number 0 : 0
Either way, `number'
now contains 3
, and in the first case,
`0'
remains unchanged,
| `0' |
|---|
| 3 earnings age if sex=="female", follow |
In the second case, `0'
is changed so that the first token is
removed:
| `0' |
|---|
| earnings age if sex=="female", follow |
By default, the token that is gotten is identified by spaces, but the
parse()
option can change that, just as it can with tokenize
.
gettoken
differs from tokenize
in that it splits off only the
first token (tokenize
continues to split the entire string into pieces),
and gettoken
puts the single results where you say (tokenize
puts
the results in `1', `2'
, ...).
The advantage of gettoken
is that you can pull off some of the string
while leaving the rest alone.
Really, this is how we should have written mytti
. The method we used
worked fine in that case, but the gettoken
method for parsing
nonstandard syntax works in all cases.
Let's pretend that we wanted to parse
. progi # # [, Constant]
The code would be
gettoken num1 0 : 0, parse(" ,")
gettoken num2 0 : 0, parse(" ,")
syntax [, Constant]
confirm number `num1'
confirm number `num2'
Let's pretend that we wanted to parse
. prog2 # [#] [varlist] [if] [, Hold Constant]
where the second number defaults to being 0 if it is not specified. The code would be
gettoken num1 0 : 0, parse(" ,")
gettoken try : 0, parse(" ,")
capture confirm number `try'
if _rc==0 {
gettoken num2 0 : 0, parse(" ,")
}
else {
local num2 0
}
syntax [varlist] [if] [, Hold Constant]
Notice what we do in line 2: we try getting the second number, but we do
not commit to it, and in particular, we do not replace `0'
with what is
left over. We then look at what we did get, and if it is a number, we go ahead
and get the token again, this time adjusting `0'
afterwards.
We can also use gettoken
to modify the syntax of
mytt
to also allow two
equal signs (==) when using the first two syntaxes. In the program mytt
the first token passed to mytt
should be a varname and the second token should be either =
or
==
. We can use the command
gettoken vn rest : 0, parse(" =")
to store the first token of macro `0'
in the macro
`vn'
and the rest of `0'
in
the macro `rest'
. Next, we can use the command
gettoken eq rest : rest, parse(" =")
to store the first token of the macro `rest'
in the macro `eq'
and
the remainder of the macro `rest'
will replace the current contents of
the macro `rest'
.
Last, if the second token `eq'
contains ==
we need
to rebuild the macro `0'
so that there is only one
=
. We need to do this because the syntax
command
will produce an error if the macro `0'
contains ==
.
The code to do this would be
if "`eq'" == "==" {
local 0 `vn' = `rest'
}
If the macro `eq'
contains something besides ==
, we don't care because
syntax
should catch any errors in `0'
.
The final code before the syntax
command of mytt
should read
/* turn "==" into "=" if needed before calling -syntax- */
gettoken vn rest : 0, parse(" =")
gettoken eq rest : rest, parse(" =")
if "`eq'" == "==" {
local 0 `vn' = `rest'
}
In the programming examples, you have seen me use the quietly
prefix
to suppress output. I have used two schemes,
quietly gen `wrk' = ...
quietly summarize ...
local mean = r(mean)
and
quietly {
gen `wrk' = ...
summarize ...
local mean = r(mean)
}
Which style you use is up to you—it makes no difference. Mostly, I prefer
the second—it's called a quietly block—because it prevents me from
forgetting to put a quietly
in front of some command that needs it. Note
that, literally, the line-by-line equivalent to my quietly block code is
quietly gen `wrk' = ...
quietly summarize ...
quietly local mean = r(mean)
The quietly
in front of local
is not necessary because
local
does not produce any output, but it would not hurt. Burying
program code in a quietly block is a convenient way of ensuring that no output
appears in the midst of calculation.
Going along with quietly
is noisily
, so you can code
quietly {
...
noisily display ...
...
}
although I do not use noisily
much. I prefer
quietly {
...
}
display ...
quietly {
...
}
If I have only one display
command in the middle, I use
the noisily
form. The choice is purely a matter of style.
As advanced programmers, however, you need to understand exactly how
quietly
and noisily
work. This also involves display
styles (colors) which play a role even on monochrome display devices.
When you say quietly
, Stata stops displaying normal
output. Normal output (by default) is defined as the display
styles "as result" (yellow) or "as text" (green).
The remaining two display
styles (colors), "as error"
(red) and "as input" (white), are considered special.
Thus if you code
quietly {
...
display "hi there"
}
the hi there
will never appear. display
produces output like any other command, so quietly
suppresses it. On the other hand, things displayed "as error" show through:
quietly {
...
display as error "you made a mistake"
}
Understand, the display
command is not acting any
differently; the difference in outcomes is a result of the display
style itself. quietly
or
not, if it is "as error" (red), it is displayed. That is why you might have a
program that reads in part
quietly {
...
regress
...
}
and if there are no prior regression estimates, the user will see
last estimates not found r(301);
The last estimates not found
message is presented with an "as
error" display style, and thus in red, so it is displayed despite the quietly
. (Then regress
produces
the 301 return code, which stops your program.)
The next thing to understand is that either you are doing things quietly
or you are not; to put it differently, there are no
degrees of quietly
:
quietly {
...
quietly regress
...
}
is no different from the above, where regress
is not
directly prefixed with quietly
. If you do something
quietly
, or quietly quietly
,
or even quietly quietly quietly
, it does not matter;
you are doing it quietly
.
noisily
apparently cancels quietly
. I say apparently because what noisily
really does is different from that, but in most
cases a noisily
cancels a quietly
. When you code
quietly {
...
noisily regress
...
}
you expect that the regression estimates will be displayed, and in most cases,
they will. But pretend that block of code shows up in program XYZsub
and that XYZsub
was invoked
by program XYZ
by the line of code quietly XYZsub
. That is,
program XYZ
...
quietly XYZsub
...
end
program XYZsub
...
quietly {
...
noisily regress
...
}
...
end
In this case, the regression estimates will not be displayed. The program
XYZsub
itself is being run quietly, and no number of noisily
s will
undo that. I will explain how Stata produces this surprising result, but first
understand that this is exactly what you would want to happen. Think of a
different context. You write a program ABC
,
program ABC
...
regress
...
end
Now, if the user types ABC
, the user should see the regression output:
. ABC
regression output appears
If the user interactively types quietly ABC
, however, the user is
demanding that no output (except error messages) appear:
. quietly ABC
nothing appears
Again, that is what happens. Despite your coding that regress
display its output in your ABC
program, the user suppressed that output. Now, if your
ABC
program happened to read
program ABC
...
quietly {
...
noisily regress
...
}
end
nothing has changed. The noisily
is intended to undo the
quietly
, but it is still the case that if the user types
. quietly ABC
the user expects to see nothing and in fact sees nothing.
noisily
produces this behavior by reestablishing the value of
quietly/noisily
when the program ABC
was invoked. noisily
does not really mean "noisily": it means "with the default output level". A
side effect of this is that if ABC
reads
program ABC
...
noisily regress
...
end
and the user types
. quietly ABC
the user still sees nothing. The noisily
does not undo the user's
quietly
: the noisily
simply reasserts the value of
noisily/quietly
that was in effect when ABC
was invoked,
which is quietly
.
This probably seems mightily confusing. To understand the effect, think of
noisily
as meaning "noisily if it makes sense, given how I was called,
and otherwise quietly":
noisily regress
means
noisily
if it makes sense ... and otherwise quietly noisily if it makes sense ... and otherwise quietlyregress
I said that there were no degrees of quietly
. That is not exactly true
because capture
has much the same effect as a quietly
in that it,
too, suppresses output. In fact, capture
suppresses output with a
vengeance. Remember that
quietly ...
suppresses the output of ... except for any output that appears "as error" (red) or "as input" (white).
capture ...
suppresses all output. quietly
suppresses all normal
output but not error messages. capture
suppresses
everything, including the error messages, because, by virtue of coding a capture
, you have indicated that you intend to handle the
error conditions yourself. What appears as an error to Stata may not really be
an error.
In a previous lecture, I coded
capture confirm string variable `varname'
if _rc==0 {
di as error "string variables not allowed"
exit 109
}
confirm string variable `varname'
checks that `varname'
is a string variable; if it is not, it issues an
error message and a nonzero return code. In what I coded, however, that error
is not a real error; the real error is, in fact, when confirm string variable `varname'
does not find an error.
Anyway, the capture
in front handled the error-message problem. The
error message that confirm string variable
felt obligated to present was
never displayed because capture
means more than quietly
; it
suppresses messages displayed "as error" (red), too.
Occasionally I have wanted to code capture
but wanted any error messages
to be displayed. I wanted to do this because I did not expect anything to go
wrong, but if something did, I wanted the user to see the error message but I
still wanted the opportunity to clean up after myself. Just think
about the question: "How could I make capture
display error messages?"
The construct is
capture noisily ...
to have the benefits of capture
, and yet still have all the output
displayed that would normally be displayed.
The constructs capture quietly
and capture noisily
appear pretty
strange. They do not seem so strange when they are used in the context of a
capture
block. You have seen capture
used to prefix a single
Stata command:
capture ...
if _rc ... {
...
}
capture
can similarly be used to prefix a group of Stata commands
capture {
...
}
if _rc!=0 ... {
...
}
The idea behind this is
capture {
group of commands that might fail}
if _rc!=0 {
cleanup code}
code if all well
Let me show you an example in a program designed to create a new variable called
result
. result
is not a temporary variable, nor is it the name of
a variable specified by the user. This program creates a variable called
result
, and let's assume that the program has already verified that the
user has no variable named result
in the dataset.
capture {
...
generate result = ...
regress ...
replace result = ... if ...
...
}
if _rc!=0 {
local rc = _rc
capture drop result
error `rc'
}
The capture { ... }
is called a capture block. All output from the
statements in the block is suppressed, but that is not what is important. If
any statement should fail—for whatever reason, including the user pressing
Break—execution of the rest of the statements in the block is aborted, the
nonzero return code is stored in _rc
, and control passes to the statement
following the close brace of the block. If the statements do not fail, they
execute normally, and 0 is stored in _rc
.
Thus, by the if _rc!=0
point, some or all of the statements in the block
have been executed. They have all been successfully executed if _rc==0
and statements were aborted otherwise. The
if _rc!=0 {
local rc = _rc
capture drop result
error `rc'
}
part of the code performs the clean up after failure. In this example, if
something went wrong, we might need to drop the variable result
.
Variable result
may not have yet been created—depending on where
the error occurred—so capture drop result
handles both cases.
Since capture
resets _rc
and we want to exit our code with the
same return code and error message that caused the problem in the first place,
we save _rc
in the local macro `rc'
first and, afterwards, code
error `rc'
to exit our program.
Capture blocks used to play an important role in Stata programming but now,
with the use of tempvar
, they play a lesser role. Another way of
coding the above problem would be
tempvar res
quietly {
...
generate `res' = ...
regress ...
replace `res' = ... if ...
...
}
...
rename `res' result
Using a temporary variable makes the problem easier to solve; if the program is
stopped along the way, Stata itself will drop the variable. Only at the end,
when we are finished, do we rename the temporary variable to its final name.
The tempvar
approach is probably better than the capture-block approach
for this problem.
Still, capture
blocks have their purpose. If we must perform any
clean up after failure, only a capture block will do.
As an example, I wrote a complicated program that had many subroutines,
and the subroutines used global macros as the primary way they communicated with
each other. Let's pretend that the command is called XYZ
and that its
syntax is
XYZ varlist
Suppose also that many of my subroutines will need to access `varlist'
.
One solution would be to pass as an argument `varlist'
to every program
that needed it, but assume that I do not want to do that. In the real case from
which I am abstracting, that was exactly the issue. I had lots of common
information I needed to share among subroutines, and it was just too much for me
to keep track of which routine needed what.
Here was my program:
program XYZ
...
capture noisily {
global XYZ_vl "`varlist'"
...
}
global XYZ_vl
exit _rc
end
Notice I used a capture noisily
block. The capture
captured the
return code, and the noisily
undid the output suppression. I put
`varlist'
in the global macro $XYZ_vl
, and then, no matter how the
program ended, I cleared XYZ_vl
. My final statement was a mere exit
_rc
because the error message, if any, had already been displayed due to the
capture noisily
. In most cases, there would be no error message and
_rc
would be 0. If something did go wrong, however, the error message
would be displayed, but my program would not stop. Instead, control would jump
to the end of the capture block where I cleared the macro and then exited
_rc
, this time with the appropriate nonzero return code.
I just said that I sometimes use global macros to communicate among my programs. I use global macros to
Global macros have names that are globally exposed, and we programmers must agree on a convention so our global names do not conflict with one another.
Moreover, the names of our programs themselves are globally exposed. How do
we ensure that I do not write a program called XYZ
and you do
not write a similarly named program?
The jargon for this problem is name collision, and the issue is how to minimize it.
The first way we minimize name collision is by using as few global names as possible. Whenever possible, follow these rules:
tempname
to obtain names for scalars and matrices.tempvar
to obtain names for variables.tempfile
to obtain filenames.That is, using local variants whenever possible is the best solution because then it does not matter if we choose the same names; Stata can keep them straight.
When local variants are not enough, there are conventions we should all follow to at least minimize the extent of the problem.
Consider the two reasons for using global macros:
Exposing common information for use while the command is active
Suppose that I am writing a complicated program to perform hierarchical stepwise estimation, and the ado-file contains lots of subroutines. Say that I find it convenient to globally expose the following information:
$HSW_dv
the dependent variable$HSW_n
the number of possible RHS variables$HSW_1
the 1st possible RHS variable$HSW_2
the 2nd possible RHS variable ...${HSW_$HSW_n}
the last possible RHS variable$HSW_if
the (optional)if exp
$HSW_in
the (optional)in range
So I just do that, and I try to make my macro names unique by mixing
capital and lowercase letters and including an underscore. (I would not name
a global macro, for instance, $IF
or $if
.)
Understand that these macros are going to be defined only while my command is active; I have no intention of leaving these macros behind after my command completes. Therefore, the first thing I need to do is find a way to clean up after myself. I write a subroutine that will clear these macros:
program ZapMacs
global HSW_dv
/* dependent variable */global HSW_if
/* optional if exp */global HSW_in
/* optional in range */forvalues i = 1/$HSW_n {
global HSW_`i'
/* `i'th RHS var */}
global HSW_n
/* number of RHS */end
Now, I have a convenient way of clearing all the global macros—I just code
ZapMacs
. The ZapMacs
program is one of the first I include in my
ado-file because it helps me keep track of the global macros I'm using. When,
as I am writing some other part of the code, I discover I need yet another
global macro, I immediately jump down and modify ZapMacs
to both document
it and clear it.
Having ZapMacs
, I ensure that there is no way out of my main program by
coding
program hsw
...
global HSW_n 0
capture noisily {
...
}
local rc = _rc
ZapMacs
exit `rc'
end
I do not like it when my programs get too indented, so my main program will quite probably read
program hsw
global HSW_n 0
capture noisily DoIt `0'
local rc = _rc
ZapMacs
exit `rc'
end
program DoIt
will contain my real code. Note the global HSW_n 0
statement. I had to include that because ZapMacs
needed HSW_n
defined, and things might go so wrong inside DoIt
that DoIt
never
gets to the place where HSW_n
is defined.
I prefer things to be more organized, and as I write this complicated hsw.ado file, other macros may need preliminary initialization. Therefore, I write my code as
| ADO-FILE:hsw.ado |
|---|
|
Keeping information between calls
We are now writing systems of commands, and unless you are writing an estimation command, there is no good solution to the problem.
For estimation commands, the solution is easy: you store what you need in
e()
. Thus if I'm writing an estimation command, and one of the things I
need to keep around is the r-squared, my estimation command reads
program myest, eclass
local options "Level(cilevel) ..."
if replay() {
if "`e(cmd)'" != "myest" { error 301 }
syntax [, `options']
}
else {
...
ereturn scalar r2 = ...
ereturn local cmd "myest"
}
...
display ... e(r2) ...
...
end
The name e(r2)
was my choice. I could have just as
well called it e(rsq)
or anything else. e(r2)
is a better choice because other estimation commands
also use that name, and users of my estimation command will probably expect to
find it under that name. If you are interested in naming conventions for saved
results, read the section titled Recommended names for saved results in [P] return.
If the command system you were writing is not an estimation command, you are just going to have to choose unlikely names for the global macros you are using to pass information and hope no one else chose the same names. In choosing these unlikely names, I offer the following guidelines:
S_
. Think of
S_
as meaning system things or Stata things. The S_ prefix has a
special meaning to Stata, especially in Stata programs written under
version 5 and older (before the introduction of r()
and e()
returned results). Many of these programs are still in use today.Other than that, any name is available to you.
The only other names that are globally exposed are the names of your programs themselves. How do we avoid giving them the same name?
The major problem is conflicting with a name used by StataCorp at some future date. We are constantly developing Stata and introducing new commands. To live with this "problem", we recommend the following:
Never choose a nice name for your programs. In particular, if
the word you have in mind appears in a dictionary or is a common
statistical term, do not use it for the name of your command.
I'm sorry, but that's what I recommend, and there is good reason for it. Everybody uses StataCorp commands, and the nice names should be reserved for the largest possible population. Even inside StataCorp, programs have to earn good names.
I am not suggesting that you name your ado-files xyzzy.ado
, I am
suggesting merely that you do not name them table.ado
, search.ado
,
decile.ado
, and the like; tbl.ado
, srch.ado
, and
dec.ado
would be better choices.
I suggest that you do not name your command using uppercase letters, because command names are reflected in ado-file names. Unix respects case, but Windows and Macintosh do not. So Unix users can adopt mixed case, but if they do, they will find it impossible to run their programs on Windows and Macintosh computers.
Nevertheless, you have seen me use mixed case for command names in these
lectures. I use mixed case for the private subroutines inside an ado-file.
Regardless of operating system, this causes Stata no problem, and I find it
convenient. I can look at a line and know whether the subroutine is in this
file. In my hsw.ado
example, I can look at the line
capture noisily DoIt `0'
and know that program DoIt
appears in the same
ado-file. It is just a personal convention.
It's okay to use nice names for your private subroutines. Those names are local, so there is no name-conflict issue.
How do we pass arguments to our programs? If you review the programs in the previous four lectures, here is what you will find:
Lecture 1 & 2: |
Lecture 3: |
Lecture 4: " |
There is a pattern here: ado-files accept "standard" syntax—meaning they
use syntax
to interpret what they receive—but subroutines sometimes
use standard syntax and sometimes positional syntax.
Subroutines are private—elegance in argument passing is not
important—and you as a programmer should do what is convenient. In fact,
if you go back over the previous lectures carefully, you will find that in every
case where a subroutine accepted standard syntax, the main program had not yet
bothered to break down what was passed into its component pieces. The main
program called the subroutine using Stata syntax because it passed the argument
`0'
. The complete parsing had not yet been done.
Once the parsing is complete, the subroutines I write tend to take positional
arguments. For instance, suppose that I am writing subroutine
XYZsub
— a subroutine of XYZ
that will appear in
XYZ.ado
—and it has three arguments:
if exp
Here is how I might code XYZsub
:
program XYZsub
args invar if outvar
...
end
Here, in my main program, is how I might call it:
program XYZ
...
tempvar result
XYZsub `1' `"`if'"' `result'
...
end
I want you to notice the double quotes around the `if'
:
XYZsub `1' `"`if'"' `result'
^^^^^^^^
They are of vital importance. Local macro `if'
might contain nothing,
and if it did, the line would read
XYZsub `1' `""' `result'
meaning `1'
is the first argument and `result'
is still the third.
If I had omitted the double quotes, `result'
would have become the second
argument, and XYZsub
would not have worked as intended.
Including double quotes on positional arguments never hurts. The line could just as well have been coded
XYZsub `"`1'"' `"`if'"' `"`result'"'
When an argument is not nothing, and when an argument does not itself contain blanks, the double quotes are not necessary.
Next, I want you to notice how I coded XYZsub
:
program XYZsub
args invar if outvar
...
end
All my subroutines that take positional arguments look like this: they first
unload the positional arguments into more readable names using the args
statement. I could code my subroutines by referring to just `1', `2',
and `3'
, but if I do, I will probably make a mistake by referring to
`3'
when I should be referring to `2'
.
Using args
also makes it easier to add arguments later, should that be
necessary.
If you write subroutines that take positional arguments, I highly recommend that you follow this practice.
Now, should you write subroutines that take positional arguments? I find it
convenient because it skips a step—that of laying out a syntax
diagram—when deciding what the syntax should be. If I were coding
XYZsub
using standard Stata syntax, should I make the syntax
. XYZsub varname [if exp], gen(newvar)
or should I make it something else? Since no one but me is ever going to see
XYZsub
, I never address this issue.
On the other hand, the minute I find a subroutine so useful that it is promoted out of the ado-file and into an ado-file of its own, I modify it to take standard syntax. If some subroutine I write is generally useful—one I would like to use in other programs I write—then it deserves my considering how the syntax should be and the safety of standard syntax. (Positional arguments are inherently dangerous because, if you are off by one, things go awry.)
Take a look at any of the real ado-files we write for Stata and you will
discover that they all start pretty much the same way. For instance,
prtesti.ado
reads
| ADO-FILE:prtesti.ado |
|---|
|
I will explain the -*! version
...- line at the top, but let's start with
the first line of the program:
version 5.0
Every program you write should start with a version statement. I omitted the line in all the examples because, today, it will not make any difference whether you include it or not and because I wanted the programs to be as short as possible so that they would fit on your screen.
I should not have omitted the line. Every program should
have that line. The version 5.0
line in prtesti.ado
tells Stata
that this program was written and debugged under Stata 5.0. Stata is constantly
under development. That line will tell future versions of Stata that, if there
have been any changes to Stata's language that might break a Stata 5.0 program,
it should temporarily undo those changes so that this program continues to work.
It transfers responsibility for ensuring that your program continues to work
with future versions of Stata from you to us. Include the line. And now, since
the most recent version of Stata is
version 10.0
, you should include the
line
version 10.0
as the first line of your programs.
The other line,
*! version 2.1.9 04sep2005
is really just a comment. All lines that start with *
are comments, but
by starting with *!
it is a special kind of comment. If you get into
Stata and type which prtesti
, you will see something like
. which prtesti
/usr/local/stata10/ado/base/p/prtesti.ado
*! version 2.2.2 17mar2005
which
tells you if an ado-file exists, and, if it exists, where it is
located; it also displays any lines that start with the characters *!
.
We use *!
to indicate when we wrote the program, although while I'm still
developing a program, I will use *!
as a way of keeping notes as well.
Right now, I'm developing a program called raz
:
. which raz
./raz.ado
*! version 1.0.0
*! bug: "raz reg y x1-x4 x5" equivalent to "raz reg (x1-x4) x5"
*! fix it.
Use *!
or not as you see fit.
Is there more to cover? Certainly, but I do not think we have failed to cover anything that you will have trouble picking up on your own. The manual does a good job of detailing what particular commands do; it does not show how the features work together, and that is what I have tried to focus on.
If you want to be an advanced programmer of Stata or any other system, you must do two things:
Step 2 is of vital importance. In NC-151, I emphasized that, for most programming, it is enough that a program work. It is not the tools that are used but step 2 that distinguishes advanced programming from application programming.
An advanced program is not merely complex. In fact, avoid unnecessary complexity. An advanced program is one that you trust in all situations, and one that you can go back and modify without trepidation and fear. I have shown how a program may not have a design at the outset, but it most certainly has one by the time it is completed. It is the design—having components that do identifiable things and do them well—that makes an advanced program.
People who write may well have larger vocabularies than the rest of us, but that is not what makes them good writers. It is that they use the grammar well. That their vocabulary is large comes about as a natural consequence of often putting together words.
I hope I have communicated the grammar of Stata and not just the vocabulary.
getrhs.ado
, syntax getrhs
, to return in r(rhs)
the names of the
independent variables from the most recently estimated model. Assume that
getrhs
will be used only after single-equation models
such as regress
, logit
, and
probit
.
Hint: the extended macro function colnames(matname)
returns the column names of a matrix or vector; see
help local
. The coefficient vector after estimation
is e(b)
. Because of an oddity of Stata, referring to
colnames(e(b)
) produces an error message. You can,
however, first copy the coefficient vector by coding
tempname b
matrix `b' = e(b)
and then use `b'
.
These column names will include _cons
if the model was
estimated with an intercept. Stata's estimation commands, if they include
_cons
, always place _cons
last.
getrhs.ado
to add a used
option. used
further
edits the returned list to exclude variables dropped in estimation.
Hint: make this determination based on the standard error _se[]
. Stata sets the recorded standard error to zero when
a variable is dropped.
todo.ado
. The syntax is
todo
lists everything you have to dotodo add text
adds text to the listtodo done
lists all tasks completed in the last 2 weekstodo done #
marks task # as donetodo undone #
marks task # as undone
todo
works by keeping a dataset of tasks. The dataset contains
no |
task number |
date |
date task assigned |
text |
text of task (up to 80 characters) |
done |
date task done |
This dataset should be called tasks.dta and should be stored in a
location of your choosing. For instance, on my Windows computer, I
decided to keep tasks.dta in C:\ado
.
Here is some output from a working todo:
. todo add do exercise 3 of lecture 5
. todo
1. 20feb2001 read nc-152 lecture 5 2. 21feb2001 do exercise 3 of lecture 5. todo done 1
. todo
2. 21feb2001 do exercise 3 of lecture 5. todo done
1. 20feb2001 21feb2001 read nc-152 lecture 5
Your program must address the following issues:
. todo
2. 21feb2001 do exercise 3 of lecture 5
1. 22feb2001 oil change on car
9. 24feb2001 print all the lectures, get notebook
3. 24feb2001 check out http://www.stata.com
Write your code so the next task number is 4.
add
, done
, and undone
.
add
may be specified as a
, ad
, or add
. done
may be specified as d
, do
, don
, or done
.
undone
may be specified as u
, und
, ...
,
undone
.Hint: despite appearances, this is not a difficult problem. It may seem artificial to you, but we at Stata actually use something similar to this and have found it very useful.
Stata stores the current date in c(current_date)
.
Do not let your inability to solve a particular problem prevent you from going
forward with the task. That is, you may not find a way to deal with the
initialization event when C:\ado\tasks.dta
(or whatever you call
it) does not exist. Fine, create the dataset by hand and complete the exercise
otherwise. Perhaps you cannot figure out how to keep task numbers from growing
without bound. Fine, let them grow. Complete the exercise and make a working
todo as well as you can.
To set you on the path to producing an elegant, modifiable program, let me emphasize that my solution to this exercise involved many subroutines. One of the programs in my ado file reads
program Add
if `"`0'"' == "" {
di as error "syntax is todo add text"
exit 198
}
preserve
UseData
GetNo
local no = r(next)
quietly {
local n = _N + 1
set obs `n'
replace no = `no' in `n'
replace date = date("$S_DATE","dmy") in `n'
replace text = `"`0'"' in `n'
}
SaveData
end


Few Words About Us
The Student Union Council is one of the most important student bodies on campus. Its main purpose is to cater to the needs of Students as a whole and bring it to the notice of the Administration be it hostel problems, sanitation on campus or even related to the mess food. It serves as a bridge between the concerned authorities and all students on campus including the higher degree ones. It was established with a vision of providing a student an easy way to bring about changes through the administration.
Eiusmod Tempor
Et harum quidem rerum facilis est et expedita distinctio. Nam libero tempore, cum soluta nobis est eligendi
Magni Dolores
Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum
Dolor Sitema
Minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat tarad limino ata
Facts
Sed ut perspiciatis unde omnis iste natus error sit voluptatem accusantium doloremque
Events
Projects
Hours Of Work
Students
Positions
Sed ut perspiciatis unde omnis iste natus error sit voluptatem accusantium doloremque
The President
The President, the Head of the Union and the Council, has the power to speak on behalf of the Union with any authority without prior consultation with other members of the Council.
The General Secretary
Below the President is the General Secretary, accountable for all the inventory of all the property owned by the Union’s organs. He takes care of the general affairs of the college and is responsible for all meetings of the Council, keeping minutes of the same
The Technical Secretary
The Technical Secretary is accountable for the technical activities and culture on the campus and organizes any intra-college technical fests.
The Sports Secretaries
The Sports Secretaries help in nurturing and developing sports culture on campus and also organize the intra college sports fests.
The Cultural Secretaries
The Cultural Secretaries co-ordinate all cultural activities of the campus and generally take up the organizing of those events along with those of the intra-college cultural fest.
The Hostel Representatives
The Hostel Representatives, elected by each hostel’s students, focus on the development along with maintenance of the hostel facilities and monitor the inventory in their respective hostels.
Team
Sed ut perspiciatis unde omnis iste natus error sit voluptatem accusantium doloremque


No, Koalas Aren’t “Functionally Extinct” – But That Doesn’t Mean They’re OK
by Thermesh VelusamyTap to listen to this article instead: 🔊 Listen to Post
It’s not been a good week for animals all over the world. At first, we heard about Iman the last Sumatran Rhino was on her deathbed, before she passed away yesterday, officially marking the extinction of the Sumatran Rhino in Malaysia. And today, due to massive bush fires that are happening in Australia which have caused the deaths of hundreds of koala bears, this species is now in real danger.
Despite many recent reports saying that koalas are going “functionally extinct“, this is not really the case. “Functionally extinct” means that there are so few breeding pairs that it would be almost impossible for them to produce a new generation. According to Christine Adams-Hosking, a biologist from the University of Queensland, there are koalas all over Australia and some of them are doing fine.
“You can’t just make that statement broad-brush.”
HOWEVER, there’s no denying the impact caused by the bush fires that have caused the death of hundreds of koalas. Koalas rely on eucalyptus leaves for their water source but the fires have destroyed their habitat alongside their food and water sources. And the number of koalas is falling because the eucalyptus forests they live in are being cut down by humans. So currently, habitat loss is their big problem.
While this may seem like a relief, the fact still stands that our activities and expansion are what is causing the number of koalas to reduce. It’s true that they aren’t “functionally extinct” but how long will it take till they are? Do we really want another Sumatran Rhino on our hands ?
Also read: Woman Risked Her Life Saving A Koala Bear From Burning to Death in Raging Bushfire


This is a paragraph.
This is another paragraph.
This is the 3rd paragraph.
动画测试
夫天地者，万物之逆旅
光阴者，百代之过客也
start:


Ledian Dibra is an associate engineer with DTE Energy. During his time at DTE, Ledian
has managed the largest summer program in the company’s history...read more >
Lizabeth Ardisana '84
CECS
Lizabeth Ardisana exemplifies Michigan excellence through her commitment to education,
entrepreneurship, and community impact...read more >
Janet Hall '90
COB
Janet Hall was the first female graduate from the College. She had a long and distinguished career
in manufacturing before her retirement in 2002...read more >
Shanelle Jackson '03
B.A.
Shanelle Jackson is a 2004 graduate from the College of Arts, Sciences, and Letters.
She is entering her third term in the Michigan House of Representatives...read more >


Overview
This is a step-by-step of how to classify thumbnail images with deep learning models using PyTorch library. All references to Python code (classes, method) can be found at this GitHub repository.
Problem
- Given two youtube channels, what thumbnail corresponds to each channel?
- It is a problem of image classification.
- We choose two YouTube channels to perform this classification: Nerdologia and Peixe Babel.
Downloading Thumbnails
We are going to use CNN (convolutional neural network) to solve the problem. The first task that we need to do is to download all thumbnails before starting building the machine learning model. This task is performed by the class YouTubeCrawler. We use pyyoutube library o access YouTube videos through an API. Besides, to do the download, you must generate the YouTube API key. See this link to get details about the API key. Then, paste your key in the constructor of YouTubeCrawler class.
class YouTubeCrawler:
def __init__(self):
self.channels = [YouTubeCrawler.PEIXE_BABEL, YouTubeCrawler.NERDOLOGIA]
api_key = '' # paste your key here
self.api = Api(api_key=api_key)
After that, by calling the fetchThumbnails
method, you will retrieve 400 thumbnails of each channel.
RGB Colors as Input Features
Each thumbnail image can be described as vectors of RGB colors. R stands for red, G stands for green, and B stands for blue. It means that a thumbnail has three vectors of numbers where each represents how much of the red, green, and blue the image has.
We extract these vectors in the fetchThumbnails
method and put them into a Thumbnail class instance.
After this extraction process, you have the original image and the RGBs vectors.
In figure 1, there is an example of a thumbnail of nerdologia channel.
| Figure 1. Example of a thumbnail |
In figure 2, we plot the same image three times, each one considering one vector of colors.
| Figure 2. Thumbnail RGB |
In figure 3, we have the RGB distribution represented by histograms.
| Figure 3. RGB vectors as histograms |
The code that plots figure 1,2,3 is in the plot method
in the class Thumbnail
.
We also save the downloaded thumbnails in the file thumbnails.npz
.
Neural Network Architecture
Once having the input features, the next step is to define the CNN architecture. We put two 2d convolutions followed by relu activation function in the convolution/activation layer. Then, we place a max-pooling 2d function in the pooling layer and flatten the vectors. Finally, we put two linear transformations in the fully connected layer that is followed by a softmax function. The softmax function will say if a thumbnail belongs to nerdologia or peixe babel channel.
In this video, the researcher Serena Yeung explains in detail how CNN works.
The class Net
describes our CNN architecture.
Training and Testing Process
Then, we start the training process by dividing our dataset into training and test set.
We instantiate the model (class Net
), feed it with data, and calculate the metrics accuracy, precision, recall, and f-measure (F1) to verify the quality of the classifications.
After each iteration, it is possible to see the model improvement in the log:
Train Epoch: 10 [530/574 (91%)] Loss: 0.619260
Train Epoch: 10 [540/574 (93%)] Loss: 0.410600
Train Epoch: 10 [550/574 (95%)] Loss: 0.341183
Train Epoch: 10 [560/574 (97%)] Loss: 0.436197
Train Epoch: 10 [228/574 (98%)] Loss: 1.016665
Test set: Average loss: 0.2127, Accuracy: 94/101 (93.07%), Precision: 0.94, Recall: 0.92, F1: 0.93
Take a look at the method train and test to see how we build the model.
Predictions
Having the model, we can start to make predictions. You inform a new thumbnail and the model will classify it into nerdologia or peixe babel (our channels).
That’s the idea of the method predict
.
Below in figure 4, there is an example of a prediction where the model classified the thumbnail as nerdologia.
| Figure 4. Prediction |
Conclusion
PyTorch is very developer-friendly. It gives you the possibility to define reusable classes/modules in an object-oriented programming manner. I find PyTorch approach flexible and powerful.


Gateway
BOT
25-Jun-18 08:29 PM
[telegram] <icantgiveyou> ask them (re @icoexplorer: Will this be sorted?)
[telegram] <rr621801> how come my seed words log me into differen agama account? I checked twice before sending my fund
[telegram] <Regnar> check to see if you included a space after the last word (re @rr621801: how come my seed words log me into differen agama account? I checked twice before sending my fund)
[telegram] <Regnar> if you are copy and pasting the seed from another file, it's likely that's the case
[telegram] <rr621801> i copied on paper
[telegram] <rr621801> and wrote word to word
[telegram] <rr621801> with and without spaces
[telegram] <rr621801> i even set a pin and pw.
[telegram] <rr621801> which was working then.
[telegram] <Acura360> Different Seed = different adress
Same Seed = same adress
[telegram] <rr621801> ticket support keeps on telling me to check spaces
[telegram] <rr621801> wow acura thanks so helpful
[telegram] <Hermes08> You sent a ticket recently I believe we are reviewing your case, you shouldn't be able to see a different address if the pin was set up
[telegram] <Acura360> (re @rr621801: wow acura thanks so helpful)
[telegram] <Regnar> try writing it out on a notepad doc or something, make sure you have all the spaces and spelling correct. then paste it in (watching for space at the end too)
[telegram] <rr621801> i tried that too
[telegram] <rr621801> there is no other way to retrieve it huh?
[telegram] <Regnar> no I don't think so. The seed is everything. you said you checked the seed before you sent funds right? (re @rr621801: there is no other way to retrieve it huh?)
[telegram] <Regnar> that the seed you were typing in was giving the same address?
[telegram] <rr621801> yes it was working
[telegram] <Hermes08> What I will do and I have seen cases like this before is trying different scenarios of errors that could have happened...
[telegram] <rr621801> does it matter if we put spaces between words?
[telegram] <Hermes08> which version of the wallet you have?
[telegram] <rr621801> latest
[telegram] <Acura360> WHICH (re @rr621801: latest)
[telegram] <Regnar> I hate to keep saying it, but there is probably a single mistake you're making in the seed. so try putting a space at the beginning, if that doesn't work, try a space at the end, etc
[telegram] <rr621801> i only set my wallet 2 days ago
[telegram] <rr621801> kk regnar it might be that
[telegram] <Hermes08> yeah if for instance when you created the seed you had 3 spaces between two words or 2 I have seen users resolve this issue by trying different scenarios
[telegram] <rr621801> like what? (re @Hermes08: What I will do and I have seen cases like this before is trying different scenarios of errors that could have happened...)
[telegram] <Hermes08> you have 24 words in your seed...there are 48 scenarios possible of having 2 or 3 spaces between one the words


Socialists and Podemos reach coalition deal in bid to form government
by EuronewsSpain's Socialists (PSOE) and left-wing Unidas Podemos (UP) have reached a coalition agreement, the parties' respective leaders announced on Tuesday.
If endorsed by parliament, it would be Spain's first coalition since the re-establishment of democracy in 1977, and its first government to have lawmakers from far-left parties.
The deal was announced by PSOE leader Pedro Sanchez — who has been Spain's acting prime minister since 2018 amid ongoing political deadlock — and UP leader Pablo Iglesias.
Spanish media reports said the agreement includes a role for Iglesias as deputy prime minister, something Sanchez had refused had to consider after a similarly inconclusive election in April.
Sanchez said he start talks with additional parties to reach a majority.
Political stalemate and the rise of the right: what you need to know about the Spanish Election
A 155-strong coalition of PSOE (120) and Unidas Podemos (35) is far from the required majority of 176 seats and the coalition would need 21 more lawmakers to form a government in the first vote.
It is expected the deal will have the backing of four additional deputies — three from new left-wing coalition Más País and one from the Regionalist Party of Cantabria.
However, The People's Party, the far-right Vox and the liberal Citizens have already said they will oppose the coalition, which means Sanchez will depend on the votes of small parties such as Basque nationalists (7) or the Catalan independentists ERC (13).
After Tuesday's announcement, Spain's Ibex-35 index slipped by around 0.9%, having previously traded 0.3% higher on the day. The market has always looked less favourably on a coalition between the Socialists and Podemos than a centre-right coalition. During its campaign, Podemos opposed the privatisation of state-controlled lender Bankia, whose shares were trading 4.5% lower.
November 10 general election results:


Critique Group Time
hide instructions
Share your names and pronouns.
Decide what order you will share in.
Pick one person to share their screen.
Click "pick a card."
Read the card aloud. (Anyone can read aloud.)
Go around the circle, giving everyone an opportunity to respond to the card.
Pick a new card and repeat until you're brought back to the main room.
pick a card


Choice Coin is a governance token that powers the Fortior Voting Protocol, which leverages Artificial Intelligence and Quantum Computing to voting algorithms that are secure, equitable, and efficient.
View our main website!Choice Coin is not just a governance token; rather, it is a representation of a world that is moving toward decentralization and openess.
Fortior's voting algorithm is simple; it allows anyone who should have a say vote without difficulty.


“Colossal, crude, terrible and sublime, Brann opened the ears of the people by the mighty power of his untamed language, by the smashing fury of his wrath of words… Waste, futile and planless, mere howling, empty, chaotic waste, for no purpose under heaven but to serve as food for idle fancies as to what might have been—such to me is the death of Brann, and my throat chokes with sorrow and my soul is sick with vain despair.” — Milo Hastings, Preface to Brann The Iconoclast


Before we begin I would like to introduce the fact that the arrangement of a HTML code can be likened to that of a ham burger. YES A HAM BURGER,you have the two buns above and below and the ham in between with some other things stuffed inside.
In the code displayed earlier,ther are three(3) important tags that we need to analyze.