Click here to go to the corresponding page for the latest version of DIALS
Small Molecule Tutorial¶
Introduction¶
While the conventional use of DIALS for small molecule data could be to run xia2, the interactive processing steps via the command line are relatively straightforward. The aim of this tutorial is to step through the process…
Data¶
The data for this tutorial are on Zenodo at https://zenodo.org/record/51405 - l-cyst_0[1-4].tar.gz i.e first four files. For this tutorial it will be assumed you have the data linked to a directory ../data - however this only matters for the import step.
Import¶
Usually DIALS processing is run on a sweep-by-sweep basis - for small molecule data however where multiple sweeps from one sample are routinely collected, with different experimental configurations, it is helpful to process all sweeps at once, therefore starting with:
dials.import allow_multiple_sweeps=True ../data/*cbf
This will create a DIALS datablock.json file with details of the 4 sweeps within it.
Spot Finding¶
This is identical to the routine usage i.e.
dials.find_spots datablock.json nproc=8
Though will of course take a little longer to work through four sweeps. Here nproc=8 was assigned (for a core i7 machine.) The spot finding is independent from sweep to sweep but the spots from all sweeps may be viewed with
dials.reciprocal_lattice_viewer datablock.json strong.pickle
Which will show how the four sweeps overlap in reciprocal space, as:
Indexing¶
Indexing here will depend on the model for the experiment being reasonably accurate. Provided that the lattices overlap in the reciprocal lattice view above, the indexing should be straightforward and will guarantee that all lattices are consistently indexed. One detail here is to split the experiments on output: this duplicates the models for the individual components rather than sharing them, which allows greater flexibility in refinement (and is critical for scan varying refinement).
dials.index datablock.json strong.pickle split_experiments=True
Without any additional input, the indexing will determine the most approproiate primitive lattice parameters and orientation which desctibe the observed reciprocal lattice positions.
Bravais Lattice Determination¶
In the single sweep tutorial the determination of the Bravais lattice is performed between indexing and refinement. This step however will only work on a single lattice at a time - therefore in this case the analysis will be performed, the results verified then the conclusion fed back into indexing as follows:
dials.refine_bravais_settings indexed.pickle experiments.json crystal_id=0
dials.refine_bravais_settings indexed.pickle experiments.json crystal_id=1
dials.refine_bravais_settings indexed.pickle experiments.json crystal_id=2
dials.refine_bravais_settings indexed.pickle experiments.json crystal_id=3
Inspect the results, conclude that the oP lattice is appropriate then assign this as a space group for indexing (in this case, P222)
dials.index datablock.json strong.pickle split_experiments=True space_group=P222
This will once again consistently index the data, this time enforcing the lattice constraints.
Refinement¶
Prior to integration we want to refine the experimental geometry and the scan varying crystal orientation and unit cell. This is performed in two steps - the first is to perform static refinement on each indexed sweep, the second to take this refined model and refine the unit cell and orientation allowing for time varying parameters:
dials.refine indexed.pickle experiments.json output.reflections=static.pickle output.experiments=static.json
dials.refine static.pickle static.json scan_varying=True
At this stage the reciprocal lattice view will show a much improved level of agreement between the indexed reflections from the four sweeps:
dials.reciprocal_lattice_viewer refined_experiments.json refined.pickle
Integration¶
At this stage the reflections may be integrated - this is run with:
dials.integrate refined.pickle refined_experiments.json nproc=8
which will integrate each sweep in sequence, again using 8 cores.
Unit Cell Refinement¶
After integration the unit cell for downstream analysis may be derived from refinement of the cell against observed two-theta angles from the reflections, across the four sweeps:
dials.two_theta_refine integrated.pickle integrated_experiments.json p4p=integrated.p4p
Here the results will be output to a p4p file for XPREP, which includes the standard uncertainties on the unit cell.
Output¶
After integration, the data should be split before exporting to a format suitable for input to XPREP or SADABS. Note that SADABS requires the batches and file names to be numbered from 1:
dials.split_experiments integrated.pickle integrated_experiments.json
dials.export format=sadabs reflections_0.pickle experiments_0.json sadabs.hklout=integrated_1.sad run=1
dials.export format=sadabs reflections_1.pickle experiments_1.json sadabs.hklout=integrated_2.sad run=2
dials.export format=sadabs reflections_2.pickle experiments_2.json sadabs.hklout=integrated_3.sad run=3
dials.export format=sadabs reflections_3.pickle experiments_3.json sadabs.hklout=integrated_4.sad run=4
If desired, p4p files for each combination of reflections_[0-3].pickle, experiments_[0-3].json could also be generated.


Module: ActiveRecord::Transactions::ClassMethods
| Relationships & Source Files | |
| Defined in: | activerecord/lib/active_record/transactions.rb |
Overview
Transactions are protective blocks where SQL statements are only permanent if they can all succeed as one atomic action. The classic example is a transfer between two accounts where you can only have a deposit if the withdrawal succeeded and vice versa. Transactions enforce the integrity of the database and guard the data against program errors or database break-downs. So basically you should use transaction blocks whenever you have a number of statements that must be executed together or not at all.
For example:
ActiveRecord::Base.transaction do
david.withdrawal(100)
mary.deposit(100)
end
This example will only take money from David and give it to Mary if neither withdrawal
nor deposit
raise an exception. Exceptions will force a ROLLBACK that returns the database to the state before the transaction began. Be aware, though, that the objects will not have their instance data returned to their pre-transactional state.
Different Active Record classes in a single transaction
Though the #transaction class method is called on some Active Record class, the objects within the transaction block need not all be instances of that class. This is because transactions are per-database connection, not per-model.
In this example a balance
record is transactionally saved even though #transaction is called on the Account
class:
Account.transaction do
balance.save!
account.save!
end
The #transaction method is also available as a model instance method. For example, you can also do this:
balance.transaction do
balance.save!
account.save!
end
Transactions are not distributed across database connections
A transaction acts on a single database connection. If you have multiple class-specific databases, the transaction will not protect interaction among them. One workaround is to begin a transaction on each class whose models you alter:
Student.transaction do
Course.transaction do
course.enroll(student)
student.units += course.units
end
end
This is a poor solution, but fully distributed transactions are beyond the scope of Active Record.
#save and #destroy are automatically wrapped in a transaction
Both #save and #destroy come wrapped in a transaction that ensures that whatever you do in validations or callbacks will happen under its protected cover. So you can use validations to check for values that the transaction depends on or you can raise exceptions in the callbacks to rollback, including after_*
callbacks.
As a consequence changes to the database are not seen outside your connection until the operation is complete. For example, if you try to update the index of a search engine in after_save
the indexer won’t see the updated record. The #after_commit callback is the only one that is triggered once the update is committed. See below.
Exception handling and rolling back
Also have in mind that exceptions thrown within a transaction block will be propagated (after triggering the ROLLBACK), so you should be ready to catch those in your application code.
One exception is the ::ActiveRecord::Rollback
exception, which will trigger a ROLLBACK when raised, but not be re-raised by the transaction block.
Warning: one should not catch ::ActiveRecord::StatementInvalid
exceptions inside a transaction block. ::ActiveRecord::StatementInvalid
exceptions indicate that an error occurred at the database level, for example when a unique constraint is violated. On some database systems, such as PostgreSQL, database errors inside a transaction cause the entire transaction to become unusable until it’s restarted from the beginning. Here is an example which demonstrates the problem:
# Suppose that we have a Number model with a unique column called 'i'.
Number.transaction do
Number.create(i: 0)
begin
# This will raise a unique constraint error...
Number.create(i: 0)
rescue ActiveRecord::StatementInvalid
# ...which we ignore.
end
# On PostgreSQL, the transaction is now unusable. The following
# statement will cause a PostgreSQL error, even though the unique
# constraint is no longer violated:
Number.create(i: 1)
# => "PG::Error: ERROR: current transaction is aborted, commands
# ignored until end of transaction block"
end
One should restart the entire transaction if an ::ActiveRecord::StatementInvalid
occurred.
Nested transactions
#transaction calls can be nested. By default, this makes all database statements in the nested transaction block become part of the parent transaction. For example, the following behavior may be surprising:
User.transaction do
User.create(username: 'Kotori')
User.transaction do
User.create(username: 'Nemu')
raise ActiveRecord::Rollback
end
end
creates both “Kotori” and “Nemu”. Reason is the ::ActiveRecord::Rollback
exception in the nested block does not issue a ROLLBACK. Since these exceptions are captured in transaction blocks, the parent block does not see it and the real transaction is committed.
In order to get a ROLLBACK for the nested transaction you may ask for a real sub-transaction by passing requires_new: true
. If anything goes wrong, the database rolls back to the beginning of the sub-transaction without rolling back the parent transaction. If we add it to the previous example:
User.transaction do
User.create(username: 'Kotori')
User.transaction(requires_new: true) do
User.create(username: 'Nemu')
raise ActiveRecord::Rollback
end
end
only “Kotori” is created. This works on MySQL and PostgreSQL. SQLite3 version >= ‘3.6.8’ also supports it.
Most databases don’t support true nested transactions. At the time of writing, the only database that we’re aware of that supports true nested transactions, is MS-SQL. Because of this, Active Record emulates nested transactions by using savepoints on MySQL and PostgreSQL. See dev.mysql.com/doc/refman/5.7/en/savepoint.html for more information about savepoints.
Callbacks
There are two types of callbacks associated with committing and rolling back transactions: #after_commit and #after_rollback.
#after_commit callbacks are called on every record saved or destroyed within a transaction immediately after the transaction is committed. #after_rollback callbacks are called on every record saved or destroyed within a transaction immediately after the transaction or savepoint is rolled back.
These callbacks are useful for interacting with other systems since you will be guaranteed that the callback is only executed when the database is in a permanent state. For example, #after_commit is a good spot to put in a hook to clearing a cache since clearing it from within a transaction could trigger the cache to be regenerated before the database is updated.
Caveats
If you’re on MySQL, then do not use Data Definition Language (DDL) operations in nested transactions blocks that are emulated with savepoints. That is, do not execute statements like ‘CREATE TABLE’ inside such blocks. This is because MySQL automatically releases all savepoints upon executing a DDL operation. When #transaction is finished and tries to release the savepoint it created earlier, a database error will occur because the savepoint has already been automatically released. The following example demonstrates the problem:
Model.connection.transaction do # BEGIN
Model.connection.transaction(requires_new: true) do # CREATE SAVEPOINT active_record_1
Model.connection.create_table(...) # active_record_1 now automatically released
end # RELEASE SAVEPOINT active_record_1
# ^^^^ BOOM! database error!
end
Note that “TRUNCATE” is also a MySQL DDL statement!
Instance Method Summary
-
#after_commit(*args, &block)
This callback is called after a record has been created, updated, or destroyed.
-
#after_create_commit(*args, &block)
Shortcut for
after_commit :hook, on: :create
. -
#after_destroy_commit(*args, &block)
Shortcut for
after_commit :hook, on: :destroy
. -
#after_rollback(*args, &block)
This callback is called after a create, update, or destroy are rolled back.
-
#after_update_commit(*args, &block)
Shortcut for
after_commit :hook, on: :update
. -
#transaction(options = {}, &block)
See the ConnectionAdapters::DatabaseStatements#transaction API docs.
Instance Method Details
#after_commit(*args, &block)
This callback is called after a record has been created, updated, or destroyed.
You can specify that the callback should only be fired by a certain action with the :on
option:
after_commit :do_foo, on: :create
after_commit :, on: :update
after_commit :do_baz, on: :destroy
after_commit :, on: [:create, :update]
after_commit :, on: [:update, :destroy]
# File 'activerecord/lib/active_record/transactions.rb', line 232
def after_commit(*args, &block) (args) set_callback(:commit, :after, *args, &block) end
#after_create_commit(*args, &block)
Shortcut for after_commit :hook, on: :create
.
# File 'activerecord/lib/active_record/transactions.rb', line 238
def after_create_commit(*args, &block) (args, on: :create) set_callback(:commit, :after, *args, &block) end
#after_destroy_commit(*args, &block)
Shortcut for after_commit :hook, on: :destroy
.
# File 'activerecord/lib/active_record/transactions.rb', line 250
def after_destroy_commit(*args, &block) (args, on: :destroy) set_callback(:commit, :after, *args, &block) end
#after_rollback(*args, &block)
This callback is called after a create, update, or destroy are rolled back.
Please check the documentation of #after_commit for options.
# File 'activerecord/lib/active_record/transactions.rb', line 258
def after_rollback(*args, &block) (args) set_callback(:rollback, :after, *args, &block) end
#after_update_commit(*args, &block)
Shortcut for after_commit :hook, on: :update
.
# File 'activerecord/lib/active_record/transactions.rb', line 244
def after_update_commit(*args, &block) (args, on: :update) set_callback(:commit, :after, *args, &block) end
#transaction(options = {}, &block)
See the ConnectionAdapters::DatabaseStatements#transaction API docs.
# File 'activerecord/lib/active_record/transactions.rb', line 211
def transaction( = {}, &block) connection.transaction(, &block) end


| Interface | Description |
|---|---|
| RMIRemoteObject |
This dummy interface is used to mark an object as RMI-compatible.
|
| RMIRemoteStub |
This interface extends the RMI Remote Object interface by forcing all stubs to have a
function that sets the reference of its own RoR.
|
| Class | Description |
|---|---|
| LocateRMIRegistry |
The class only contains one method, which returns the RMI Registry Client for a given endpoint if
valid.
|
| RemoteObjectRef |
This class holds the Remote Object Reference, which contains the hostname and port number for the
RMI Service Host.
|
| RMIInvocationPkg |
This class contains all necessary information for an RMI function call, including the Object
Reference, method name, and the argument list.
|
| RMIRegistry |
This class contains the entry point of RMI Registry Server.
|
| RMIRegistryClient |
This class provides methods to interact with an RMI Registry Server from a client's perspective.
|
| RMIRegistryServer |
Runnable class for RMI Registry Server.
|
| RMIServer |
This class contains the entry point for an RMI Server.
|
| RMIServerCLI |
This class contains the Runnable for the RMI Registry Command-Line Interface.
|
| RoREntryTable |
This class holds all Remote Object References within a RMI Server Host.
|
| Exception | Description |
|---|---|
| RMIInvocationException |
This Exception class is the wrapper for all exceptions during RMI Invocations.
|


subject¶
Specifies the subject to study in the postprocessing commands, such as plotting and exporting.
Syntax¶
subject = index
subject = all
subject = average
where index is a scalar expression evaluating to a positive integer (1-based index into the simulated subjects).
Description¶
subject = index
uses only the subject with index index
in the subsequent postprocessing commands.
subject = all
uses all subjects in the subsequent postprocessing commands. In plotting commands,
this will produce one line per subject.
subject = average
will compute the average value of all subjects in the subsequent postprocessing commands. In plotting commands, this will produce one single line, representing the average of all subjects.
Dependencies¶
When specified as an index, this index must be \(\ge 1\) and \(\le n\) where \(n\) is the value of
n_subjects
.
Examples¶
n_subjects = 10
subject = all
will make the plotting commands render 10 lines, one per subject.
n_subjects = 10
subject = all
will make the plotting commands render one line showing the average of all subjects.
@variables p = 5
n_subjects = 10
subject = p + 1
will make the plotting commands render one line showing the values for subject number 6.


Did someone say BULK?!
We provide wholesale and office coffee services, including equipment and training. Whether you're looking for a long-term coffee provider or cafe service for a large event, we're happy to help!
Please contact us directly for more information and pricing.


Rewrite Domains
Configuration
The 'Rewrite Domains' filter is enabled by specifying:
- Apache:
ModPagespeedEnableFilters rewrite_domains
- Nginx:
pagespeed EnableFilters rewrite_domains;
in the configuration file. The filter is not enabled by default and should be added manually to the configuration file.
By default rewrite_domains
only applies to resources, but it can
optionally rewrite domains in hyperlinks as well:
- Apache:
ModPagespeedDomainRewriteHyperlinks on
- Nginx:
pagespeed DomainRewriteHyperlinks on;
Description
This filter applies all domain mapping
directives specified in pagespeed.conf to web resources that
are not otherwise rewritten by PageSpeed. For
example, if a resource is not cacheable, or filters which affect the
resource are turned off, then it still may be desirable to apply
domain sharding. (Domain sharding is not applied to hyperlinks even if DomainRewriteHyperlinks
is enabled.)
Requirements
It is the responsibility of the site administrator to set up the shard entries in their DNS or CNAME configuration. Also, please see the note about the servers for rewrite domains — this applies to sharded domains as well. The sharded domains must have access to the same content as the original domain.
Risks
It is the responsbility of the site administrator to ensure that moving resources onto domains does not create a security vulnerability. In particular, if the target domain has cookies, then any JavaScript loaded from a resource moved to a domain with cookies will gain access to those cookies. In general, moving resources to a cookieless domain is a great way to improve security. Be aware that CSS can load JavaScript in certain environments.


Math Expression Write a math expression with subtraction operators only, like " 8-5-6-4-1- " or " 7-4-1-2-5-8-9-6-3-" Note: add a subtraction operator to the end of the math expression .


Introduction
Best "permanent" URL to get to this page:
http://www.kagi.com/weyer/#Unzip
UnzipBMP is a Newt's Cape 2.x-only plugin
that extracts files from .zip files, and converts and displays .bmp image files;
(these two file types are currently combined in one plugin for my maintenance and documentation convenience).
If you find problems, email Steve
with details, including URLs of web page or specific files,
and version of Newt's Cape, UnzipBMP and MessagePad. Of course, I'd also like to hear of successes...
Features
- requirements
- Newt's Cape to use UnzipBM and other helperApps
- zip: requires Zlib.pkg library from Paul Guyot,
if you want to access compressed(Deflated only) files
- zip: helperApps; currently, the file extension is used to determine the MIME type, and thus the helperApp;
here are currently handled file extensions, MIME types and associated helperApps
and actions; you can find out which are installed in Newt's Cape by selecting View/Helpers;
let me know which other file types and apps could be added:
- .htm, .html: text/html
- Newt's Cape: displays document
- .txt: text/plain
- Newt's Cape: displays document
- .gif: image/gif
- Newt's Cape: displays image
- .jpg, .jpeg: image/jpeg
- JPEG: Newt's Cape displays image
- .bmp, .wbm: image/bmp
- UnzipBMP: Newt's Cape displays image [if b&w or 4- or 16-grays]
- .pkg: application/x-newton-compatible-pkg
- Newt's Cape: installs .pkg or stores in Inbox; note: there's a bug "No" still installs .pkg
- .pdb: application/x-pilot-pdb
- .prc: application/x-pilot-prc
- PilotDoc: Newt's Cape displays book (only for typeID: TEXtREAd or vIMGView)
- .wav: audio/x-wav
- .aif, .aiff: audio/x-aiff
- .au: audio/basic
- AudInbox: SimpleMail plays sound
- .mod: www/unknown
- ModSaver: ModPlayer plays sound (or NewtPack saves sound)
- .class, .pdb, .wrp, .jar
- WabaTester: creates local Waba app with Java classes
- note: this needs to be revised to use UnzipBMP; it may conflict with PilotDoc
- transfer
- http
- Newt's Cape: select or enter a URL to download
.zip or .bmp files directly to Newton
- Inbox
- requires Newt's Cape 2.0x-2(or later);
select file via Newt's Cape:File:Open Inbox or Inbox:Newt's Cape Open
- EE Transfer:
assumes ZModem or transfer protocol that preserves file name and binary content
- SimpleMail 4.0:
select the .zip or .bmp file via Part picker (ignore 'no stationery' warning)
- file extensions: MIME types
- .zip: application/zip, application/zip-compressed
- .bmp, .wbm: image/bmp
- (others?)
- file format & size
- if you transfer file yourself, make sure it's binary (Newt's Cape will download .zip and .bmp as binary automatically);
make sure you have enough room, both for the file, and for any file you Extract
- how to use
- download a .zip or .bmp file with Newt's Cape, or use Newt's Cape:Open on an Inbox item
- .bmp
- the image should automatically convert and display in Newt's Cape
- note: it is not currently saved in Newt's Cape's Image cache
- .zip
- an overview appears showing the .zip file contents
- (window may be somewhat smaller if NIE is connected to avoid system bugs)
- use scroll arrows to move through a large .zip file
- [optional] Save .zip to Inbox (for later use)
- select files(s); numbers of total items, current range, and selected items appear in title
- note: checkboxes are shown only for uncompressed or Deflated files (not Imploded or other file types; or directories);
for Deflated files, Zlib.pkg library must be installed
- Extract files
- note: if you want to Extract more than one file, and a helperApp can handle only one file at a time,
you may wish to Save to Inbox first, and then Extract one file at a time; e.g., if you Extract two .htm files, only the second will be displayed (or maybe an error will occur?);
it should be ok to Extract at once: .pkg (Newt's Cape install), .bmp (Newt's Cape display), .wav (SimpleMail play)
Other Approaches to .zip files
- decompress on your desktop
- server-based conversion;
instructions for www.palmzip.com may be found on the PilotDoc plugin page
- 1.0b (8 Nov 01):
- .zip: added "application/x-zip-compressed" MIME type
- .bmp: handles 4-grays (2 bits-per-pixel), 16-grays (4 bpp); error if greater, e.g., 256 colors
- tap on Extension icon to see brief description and URL
- 1.0a (6 Nov 01): first version
Issues
- extracting multiple .pkg might be a problem -- perhaps requiring a fix to Newt's Cape (or putting .pkg installer code in UnzipBMP)
- Color mapping isn't handled -- if someone wants to help me figure out how to map pixel to color(rgb) values,
I know how to map those to gray.
Download
© Copyright 2001-2002, S. Weyer. All Rights Reserved Worldwide.


Since bblog 3.5 the plotted graphics for the weapon spreaddata are no longer working.
It seems to be an issue with not defined variables starting with "minSpreadBase". See the following error message from Chrome: "Uncaught ReferenceError: minSpreadBase is not defined".
Post edited 1 x times, last by
xXJumpyXx
Helping Hand
xXJumpyXx
Helping Hand
Seems to not be working as it should on Firefox for me. No bullet spread is showing up, the background is always white.
That is if we're talking about the same thing:


A data-join is followed by operations on the enter, update and exit selections. Entering elements are green, while updating elements are black. Exiting elements are removed immediately, so they're invisible. This example does not use a key function for the data-join, so elements may change their associated letter. Entering elements are always added to the end: when the new data has more letters than the old data, new elements are entered to display the new letters. Likewise, exiting letters are always removed from the end when the new data has fewer letters than the old data.


Before we go any further, let’s take a look at the Python docs for the version of Python ArcGIS 10.1 uses, 2.7.2:
http://docs.python.org/release/2.7.2/
Let’s look at the os and urllib modules, which we will be using in this section.
>>> import urllib
>>> url = "http://www.fhwa.dot.gov/bridge/nbi/2012/delimited/DC12.txt"
>>> f = r"C:\Users\class5user\ar-gis-python\outputs\dc.csv"
>>> urllib.urlretrieve(url, f)
If files are sitting out on a server somewhere in a fairly rigid folder structure, it’s relatively easy to fetch those files. The Libre Map Project hosts the 24K USGS DRGs for all 50 states for FREE on archive.org. We are interested in the Arkansas DRGs.
On the Arkansas data page, hover over some of the links for the TIFF, TFW, and FGD files. See a pattern there?
http://www.archive.org/download/usgs_drg_ ar _ 35 094 _ a2 / o 35094 a2 .tif
This breaks down to:
base_url state _ deg lat deg lon _ map index no / category deg lat deg lon map index no .file ext
Note
This USGS standard explains the above nicely: http://topomaps.usgs.gov/drg/drg_name.html
This quad information is available in the attributes of the 24K USGS quadrangle footprint that is readily available on the web. I got mine from the USDA NRCS Data Gateway. Order by State | Arkansas | Map Indexes | Quadrangle Index 1:24,000. You have this in the shapefiles directory as quads24k_a_ar.shp. Load it into ArcMap and look at the attribute table.
Now open up 24K_Quads.xlsx from the inputs directory, in Excel. Let’s build our formula to in turn create our list of drgs to fetch.
Look in Cell B16
Import modules we will use:
import urllib
import os
Paste in your DRG list:
drg_list = [['o36094a2', 'Fayetteville', 'AR'],
['o36094b1', 'Sonora', 'AR'],
['o36094a1', 'Elkins', 'AR'],
['o36094b2', 'Springdale', 'AR']]
Create a list of extensions, these are the 3 filetypes we will fetch for each quad:
exts = ['tif', 'tfw', 'fgd']
Setup the base_url:
base_url= "http://www.archive.org/download/usgs_drg_"
Start a for loop to iterate through each nested list (each DRG) in drg_list:
for drg in drg_list:
Start another for loop nested in the previous loop to get all 3 filetypes for each DRG (note it is indented 4 spaces!):
for ext in exts:
Let’s play around with drg_list before we move on. Copy drg_list above and paste it into the ArcGIS Python window.
Type the following commands to get a feel for how lists and slicing work:
>>> for drg in drg_list:
print drg
>>> for drg in drg_list:
print type(drg)
>>> for drg in drg_list:
for each in drg:
print each
>>> drg_list[0]
>>> drg_list[2]
>>> drg_list[0][1:6]
>>> drg_list[0][6:]
Now let’s build on list indexing and slicing and construct our url for fetching the data we want (note that this and all following lines are indented 8 spaces!). This takes parts of the quad info and builds the url for each DRG on the fly:
full_url = (base_url + drg[2].lower() + '_' + drg[0][1:6] +
'_' + drg[0][6:] + "/" + drg[0] + '.' + ext)
We need to have a local file and file path to store the fetched file in.
local_file = os.path.join(os.path.dirname(os.path.split(os.path.abspath(__file__))[0]),
"outputs", drg[0] + '.' + ext)
print local_file
Call urllib.urlretrieve method and fetch the file:
urllib.urlretrieve(full_url, local_file)
Let’s figure out what all the os.whatever stuff was in the script we just wrote by deconstructing it.
In Windows Explorer, navigate to the source\code directory. Create a new file there call os-stuff.py. Right-click on the file and select “Edit in IDLE”.
Enter the following in os-stuff.py:
import os
print "os.path.abspath:", os.path.abspath(__file__)
Save the file. Leave it open in IDLE. Open up a Windows command prompt it you don’t have one open already. Navigate to the source\code directory and enter:
C:\Users\class5user\argis-python-2013\source\code>python os-stuff.py
On line 3, enter:
print "os.path.split:", os.path.split(os.path.abspath(__file__))
Save and run again.
One line 4, enter:
print "os.path.split[0]:", os.path.split(os.path.abspath(__file__))[0]
Save and run again.
On line 5, enter:
print "os.path.dirname:", os.path.dirname(os.path.split(os.path.abspath(__file__))[0])
Save and run again.
One line 6-7 enter:
print "os.path.join:", os.path.join(os.path.dirname(os.path.split(os.path.abspath(__file__))[0]),
"outputs")
Save and run for the last time.
import urllib
import os
drg_list = [['o36094a2', 'Fayetteville', 'AR'],
['o36094b1', 'Sonora', 'AR'],
['o36094a1', 'Elkins', 'AR'],
['o36094b2', 'Springdale', 'AR']]
exts = ['tif', 'tfw', 'fgd']
base_url= "http://www.archive.org/download/usgs_drg_"
for drg in drg_list:
for ext in exts:
# http://www.archive.org/download/usgs_drg_ar_35094_d2/o35094d2.tif
full_url = (base_url + drg[2].lower() + '_' + drg[0][1:6] +
'_' + drg[0][6:] + "/" + drg[0] + '.' + ext)
local_file = os.path.join(os.path.dirname(os.path.split(os.path.abspath(__file__))[0]),
"outputs", drg[0] + '.' + ext)
print local_file
urllib.urlretrieve(full_url, local_file)
Apply the principles of lists, iteration, and building of a url pattern we just learned to the first example of the NBI bridge data. Is there a way to fetch multiple states by using a list somehow?
Note
Use small states like RI, DE, and the District of Columbia, that way the files you fetch are smaller.
import urllib
import os
base_url = "http://www.fhwa.dot.gov/bridge/nbi/2012/delimited/"
states = ["RI", "DE", "DC"]
for state in states:
url = base_url + state + "12.txt"
local_file = os.path.join(os.path.dirname(os.path.split(os.path.abspath(__file__))[0]),
"outputs", state + "12.txt")
urllib.urlretrieve(url, local_file)


takepic
Syntax
takepic (x1, y1, x2, y2 : int, var buffer : array 1 .. * of int)
Description
The takepic procedure is used to record the pixel values in a
rectangle, with left bottom and right corners of (x1,
y1) and (x2, y2), in the buffer array. This
requires a sufficiently-large buffer (see sizepic ).
The drawpic procedure is used to make copies of the recorded
rectangle on the screen.
Example
After drawing a happy face, this program copies the face to a new
location.
Details
The integer values that takepic places in the buffer can be
read or written (using the read and write statements).
Unfortunately, if a value happens to be the pattern used to represent
the uninitialized value (the largest negative number the hardware can
represent) assignment (by:=) and put of the individual integer
values in the buffer will fail.
The screen should be in a "graphics" mode. See the
setscreen procedure for details. If the screen is not in a
"graphics" mode, it will automatically be set to
"graphics" mode.
Details
The drawpic, takepic, and sizepic subprograms
have been made obsolete by the subprograms
Pic.Draw and
Pic.New of the
Pic module. Users are
strongly suggested to use those routines instead. The
Pic module also contains
subprograms to load images from files.
See also
sizepic and
drawpic. See also
setscreen,
maxx,
maxy,
drawdot,
drawline,
drawbox, and
drawoval.
See also predefined unit Pic.


Sage references the AForge.Imaging library, which, in addition to the imaging facilities already built-in in .NET framework, enables advanced image processing capabilities.
Using several file handlers, Sage enables inline manipulation of any image in a project if accessed through HTTP, simply by using additional query string parameters for the image request. Multiple filters can be specified in the same query string, and they all get applied, in the order of parameter appearance within the query.
The filters get applied before the processed image is returned back to the client. The generated file is then cached indefinitely for that query string combination, and will not be generated again unless the source file is updated.
With URL rewriting, the parameters can be hidden, and restriction be placed on which parameters are used and within which ranges.