Pig Latin Reference Manual 1
Overview
Use this manual together with Pig Latin Reference Manual 2.
Also, be sure to review the information in the Pig Cookbook.
Pig Latin Statements
A Pig Latin statement is an operator that takes a relation as input and produces another relation as output. (This definition applies to all Pig Latin operators except LOAD and STORE which read data from and write data to the file system.) Pig Latin statements can span multiple lines and must end with a semi-colon ( ; ). Pig Latin statements are generally organized in the following manner:
-
A LOAD statement reads data from the file system.
-
A series of "transformation" statements process the data.
-
A STORE statement writes output to the file system; or, a DUMP statement displays output to the screen.
Running Pig Latin
You can execute Pig Latin statements:
- Using grunt shell or command line
- In mapreduce mode or local mode
- Either interactively or in batch
Note that Pig now uses Hadoop's local mode (rather than Pig's native local mode).
A few run examples are shown here; see Pig Setup for more examples.
Grunt Shell - interactive, mapreduce mode (because mapreduce mode is the default you do not need to specify)
$ pig ... - Connecting to ... grunt> A = load 'data'; grunt> B = ... ;
Grunt Shell - batch, local mode (see the exec and run commands)
$ pig -x local grunt> exec myscript.pig; or grunt> run myscript.pig;
Command Line - batch, mapreduce mode
$ pig myscript.pig
Command Line - batch, local mode mode
$ pig -x local myscript.pig
In general, Pig processes Pig Latin statements as follows:
-
First, Pig validates the syntax and semantics of all statements.
-
Next, if Pig encounters a DUMP or STORE, Pig will execute the statements.
In this example Pig will validate, but not execute, the LOAD and FOREACH statements.
A = LOAD 'student' USING PigStorage() AS (name:chararray, age:int, gpa:float); B = FOREACH A GENERATE name;
In this example, Pig will validate and then execute the LOAD, FOREACH, and DUMP statements.
A = LOAD 'student' USING PigStorage() AS (name:chararray, age:int, gpa:float); B = FOREACH A GENERATE name; DUMP B; (John) (Mary) (Bill) (Joe)
See Multi-Query Execution for more information on how Pig Latin statements are processed.
Pig Latin Scripts
See the following:
- Running the Pig Scripts in Local Mode
- Running the Pig Scripts in MapReduce Mode
- Multi-Query Execution
Pig supports running scripts that are stored in HDFS, Amazon S3, or other distributed file systems (also see REGISTER for information about Jar files). The script's full location URI is required. For example, to run a Pig script on HDFS, do the following:
java -cp pig.jar org.apache.pig.Main hdfs://nn.mydomain.com:9020/myscripts/script.pig
Using Comments in Pig Latin Scripts
If you place Pig Latin statements in a script, the script can include comments.
-
For multi-line comments use /* …. */
-
For single line comments use --
/* myscript.pig My script includes three simple Pig Latin Statements. */ A = LOAD 'student' USING PigStorage() AS (name:chararray, age:int, gpa:float); -- load statement B = FOREACH A GENERATE name; -- foreach statement DUMP B; --dump statement
Retrieving Pig Latin Results
Pig Latin includes operators you can use to retrieve the results of your Pig Latin statements:
-
Use the DUMP operator to display results to a screen.
-
Use the STORE operator to write results to a file on the file system.
Storing Intermediate Data
Pig stores the intermediate data generated between MapReduce jobs in a temporary location on HDFS. This location must already exist on HDFS prior to use. This location can be configured using the pig.temp.dir property. The property's default value is "/tmp" which is the same as the hardcoded location in Pig 0.7.0 and earlier versions.
Debugging Pig Latin
Pig Latin includes operators that can help you debug your Pig Latin statements:
-
Use the DESCRIBE operator to review the schema of a relation.
-
Use the EXPLAIN operator to view the logical, physical, or map reduce execution plans to compute a relation.
-
Use the ILLUSTRATE operator to view the step-by-step execution of a series of statements.
Complex Pig scripts often generate many MapReduce jobs. To help you debug a script, Pig prints a summary of the execution that shows which relations (aliases) are mapped to each MapReduce job.
JobId Maps Reduces MaxMapTime MinMapTIme AvgMapTime MaxReduceTime MinReduceTime AvgReduceTime Alias Feature Outputs job_201004271216_12712 1 1 3 3 3 12 12 12 B,C GROUP_BY,COMBINER job_201004271216_12713 1 1 3 3 3 12 12 12 D SAMPLER job_201004271216_12714 1 1 3 3 3 12 12 12 D ORDER_BY,COMBINER hdfs://wilbur20.labs.corp.sp1.yahoo.com:9020/tmp/temp743703298/tmp-2019944040,
Working with Data
Pig Latin allows you to work with data in many ways. In general, and as a starting point:
-
Use the FILTER operator to work with tuples or rows of data. Use the FOREACH operator to work with columns of data.
-
Use the GROUP operator to group data in a single relation. Use the COGROUP and JOIN operators to group or join data in two or more relations.
-
Use the UNION operator to merge the contents of two or more relations. Use the SPLIT operator to partition the contents of a relation into multiple relations.
Case Sensitivity
The names (aliases) of relations and fields are case sensitive. The names of Pig Latin functions are case sensitive. The names of parameters (see Parameter Substitution) and all other Pig Latin keywords are case insensitive.
In the example below, note the following:
-
The names (aliases) of relations A, B, and C are case sensitive.
-
The names (aliases) of fields f1, f2, and f3 are case sensitive.
-
Function names PigStorage and COUNT are case sensitive.
-
Keywords LOAD, USING, AS, GROUP, BY, FOREACH, GENERATE, and DUMP are case insensitive. They can also be written as load, using, as, group, by, etc.
-
In the FOREACH statement, the field in relation B is referred to by positional notation ($0).
grunt> A = LOAD 'data' USING PigStorage() AS (f1:int, f2:int, f3:int); grunt> B = GROUP A BY f1; grunt> C = FOREACH B GENERATE COUNT ($0); grunt> DUMP C;
Dynamic Invokers
Often you may need to use a simple function that is already provided by standard Java libraries, but for which a UDF has not been written. Dynamic Invokers allow you to refer to Java functions without having to wrap them in custom Pig UDFs, at the cost of doing some Java reflection on every function call.
... DEFINE UrlDecode InvokeForString('java.net.URLDecoder.decode', 'String String'); encoded_strings = LOAD 'encoded_strings.txt' as (encoded:chararray); decoded_strings = FOREACH encoded_strings GENERATE UrlDecode(encoded, 'UTF-8'); ...
Currently, Dynamic Invokers can be used for any static function that accepts no arguments or some combination of strings, ints, longs, doubles, floats, or arrays of same, and returns a string, an int, a long, a double, or a float. Primitives only for the numbers, no capital-letter numeric classes as arguments. Depending on the return type, a specific kind of Invoker must be used: InvokeForString, InvokeForInt, InvokeForLong, InvokeForDouble, or InvokeForFloat.
The DEFINE statement is used to bind a keyword to a Java method, as above. The first argument to the InvokeFor* constructor is the full path to the desired method. The second argument is a space-delimited ordered list of the classes of the method arguments. This can be omitted or an empty string if the method takes no arguments. Valid class names are string, long, float, double, and int. Invokers can also work with array arguments, represented in Pig as DataBags of single-tuple elements. Simply refer to string[], for example. Class names are not case-sensitive.
The ability to use invokers on methods that take array arguments makes methods like those in org.apache.commons.math.stat.StatUtils available (for processing the results of grouping your datasets, for example). This is helpful, but a word of caution: the resulting UDF will not be optimized for Hadoop, and the very significant benefits one gains from implementing the Algebraic and Accumulative interfaces are lost here. Be careful if you use invokers this way.
Memory Management
Pig allocates a fix amount of memory to store bags and spills to disk as soon as the memory limit is reached. This is very similar to how Hadoop decides when to spill data accumulated by the combiner.
The amount of memory allocated to bags is determined by pig.cachedbag.memusage; the default is set to 10% of available memory. Note that this memory is shared across all large bags used by the application.
Multi-Query Execution
With multi-query execution Pig processes an entire script or a batch of statements at once.
Turning it On or Off
Multi-query execution is turned on by default. To turn it off and revert to Pig's "execute-on-dump/store" behavior, use the "-M" or "-no_multiquery" options.
To run script "myscript.pig" without the optimization, execute Pig as follows:
$ pig -M myscript.pig or $ pig -no_multiquery myscript.pig
How it Works
Multi-query execution introduces some changes:
-
For batch mode execution, the entire script is first parsed to determine if intermediate tasks can be combined to reduce the overall amount of work that needs to be done; execution starts only after the parsing is completed (see the EXPLAIN operator and the exec and run commands).
-
Two run scenarios are optimized, as explained below: explicit and implicit splits, and storing intermediate results.
Explicit and Implicit Splits
There might be cases in which you want different processing on separate parts of the same data stream.
Example 1:
A = LOAD ... ... SPLIT A' INTO B IF ..., C IF ... ... STORE B' ... STORE C' ...
Example 2:
A = LOAD ... ... B = FILTER A' ... C = FILTER A' ... ... STORE B' ... STORE C' ...
In prior Pig releases, Example 1 will dump A' to disk and then start jobs for B' and C'. Example 2 will execute all the dependencies of B' and store it and then execute all the dependencies of C' and store it. Both are equivalent, but the performance will be different.
Here's what the multi-query execution does to increase the performance:
-
For Example 2, adds an implicit split to transform the query to Example 1. This eliminates the processing of A' multiple times.
-
Makes the split non-blocking and allows processing to continue. This helps reduce the amount of data that has to be stored right at the split.
-
Allows multiple outputs from a job. This way some results can be stored as a side-effect of the main job. This is also necessary to make the previous item work.
-
Allows multiple split branches to be carried on to the combiner/reducer. This reduces the amount of IO again in the case where multiple branches in the split can benefit from a combiner run.
Storing Intermediate Results
Sometimes it is necessary to store intermediate results.
A = LOAD ... ... STORE A' ... STORE A''
If the script doesn't re-load A' for the processing of A the steps above A' will be duplicated. This is a special case of Example 2 above, so the same steps are recommended. With multi-query execution, the script will process A and dump A' as a side-effect.
Store vs. Dump
With multi-query exection, you want to use STORE to save (persist) your results. You do not want to use DUMP as it will disable multi-query execution and is likely to slow down execution. (If you have included DUMP statements in your scripts for debugging purposes, you should remove them.)
DUMP Example: In this script, because the DUMP command is interactive, the multi-query execution will be disabled and two separate jobs will be created to execute this script. The first job will execute A > B > DUMP while the second job will execute A > B > C > STORE.
A = LOAD 'input' AS (x, y, z); B = FILTER A BY x > 5; DUMP B; C = FOREACH B GENERATE y, z; STORE C INTO 'output';
STORE Example: In this script, multi-query optimization will kick in allowing the entire script to be executed as a single job. Two outputs are produced: output1 and output2.
A = LOAD 'input' AS (x, y, z); B = FILTER A BY x > 5; STORE B INTO 'output1'; C = FOREACH B GENERATE y, z; STORE C INTO 'output2';
Error Handling
With multi-query execution Pig processes an entire script or a batch of statements at once. By default Pig tries to run all the jobs that result from that, regardless of whether some jobs fail during execution. To check which jobs have succeeded or failed use one of these options.
First, Pig logs all successful and failed store commands. Store commands are identified by output path. At the end of execution a summary line indicates success, partial failure or failure of all store commands.
Second, Pig returns different code upon completion for these scenarios:
-
Return code 0: All jobs succeeded
-
Return code 1: Used for retrievable errors
-
Return code 2: All jobs have failed
-
Return code 3: Some jobs have failed
In some cases it might be desirable to fail the entire script upon detecting the first failed job. This can be achieved with the "-F" or "-stop_on_failure" command line flag. If used, Pig will stop execution when the first failed job is detected and discontinue further processing. This also means that file commands that come after a failed store in the script will not be executed (this can be used to create "done" files).
This is how the flag is used:
$ pig -F myscript.pig or $ pig -stop_on_failure myscript.pig
Backward Compatibility
Most existing Pig scripts will produce the same result with or without the multi-query execution. There are cases though where this is not true. Path names and schemes are discussed here.
Any script is parsed in it's entirety before it is sent to execution. Since the current directory can change throughout the script any path used in LOAD or STORE statement is translated to a fully qualified and absolute path.
In map-reduce mode, the following script will load from "hdfs://<host>:<port>/data1" and store into "hdfs://<host>:<port>/tmp/out1".
cd /; A = LOAD 'data1'; cd tmp; STORE A INTO 'out1';
These expanded paths will be passed to any LoadFunc or Slicer implementation. In some cases this can cause problems, especially when a LoadFunc/Slicer is not used to read from a dfs file or path (for example, loading from an SQL database).
Solutions are to either:
-
Specify "-M" or "-no_multiquery" to revert to the old names
-
Specify a custom scheme for the LoadFunc/Slicer
Arguments used in a LOAD statement that have a scheme other than "hdfs" or "file" will not be expanded and passed to the LoadFunc/Slicer unchanged.
In the SQL case, the SQLLoader function is invoked with 'sql://mytable'.
A = LOAD 'sql://mytable' USING SQLLoader();
Implicit Dependencies
If a script has dependencies on the execution order outside of what Pig knows about, execution may fail.
Example
In this script, MYUDF might try to read from out1, a file that A was just stored into. However, Pig does not know that MYUDF depends on the out1 file and might submit the jobs producing the out2 and out1 files at the same time.
... STORE A INTO 'out1'; B = LOAD 'data2'; C = FOREACH B GENERATE MYUDF($0,'out1'); STORE C INTO 'out2';
To make the script work (to ensure that the right execution order is enforced) add the exec statement. The exec statement will trigger the execution of the statements that produce the out1 file.
... STORE A INTO 'out1'; EXEC; B = LOAD 'data2'; C = FOREACH B GENERATE MYUDF($0,'out1'); STORE C INTO 'out2';
Example
In this script, the STORE/LOAD operators have different file paths; however, the LOAD operator depends on the STORE operator.
A = LOAD '/user/xxx/firstinput' USING PigStorage(); B = group .... C = .... agrregation function STORE C INTO '/user/vxj/firstinputtempresult/days1'; .. Atab = LOAD '/user/xxx/secondinput' USING PigStorage(); Btab = group .... Ctab = .... agrregation function STORE Ctab INTO '/user/vxj/secondinputtempresult/days1'; .. E = LOAD '/user/vxj/firstinputtempresult/' USING PigStorage(); F = group .... G = .... aggregation function STORE G INTO '/user/vxj/finalresult1'; Etab =LOAD '/user/vxj/secondinputtempresult/' USING PigStorage(); Ftab = group .... Gtab = .... aggregation function STORE Gtab INTO '/user/vxj/finalresult2';
To make the script works, add the exec statement.
A = LOAD '/user/xxx/firstinput' USING PigStorage(); B = group .... C = .... agrregation function STORE C INTO '/user/vxj/firstinputtempresult/days1'; .. Atab = LOAD '/user/xxx/secondinput' USING PigStorage(); Btab = group .... Ctab = .... agrregation function STORE Ctab INTO '/user/vxj/secondinputtempresult/days1'; EXEC; E = LOAD '/user/vxj/firstinputtempresult/' USING PigStorage(); F = group .... G = .... aggregation function STORE G INTO '/user/vxj/finalresult1'; .. Etab =LOAD '/user/vxj/secondinputtempresult/' USING PigStorage(); Ftab = group .... Gtab = .... aggregation function STORE Gtab INTO '/user/vxj/finalresult2';
Optimization Rules
Pig supports various optimization rules.
ImplicitSplitInserter
Status: Mandatory
SPLIT is the only operator that models multiple outputs in Pig. To ease the process of building logical plans, all operators are allowed to have multiple outputs. As part of the optimization, all non-split operators that have multiple outputs are altered to have a SPLIT operator as the output and the outputs of the operator are then made outputs of the SPLIT operator. An example will illustrate the point. Here, a split will be inserted after the LOAD and the split outputs will be connected to the FILTER (b) and the COGROUP (c).
A = LOAD 'input'; B = FILTER A BY $1 == 1; C = COGROUP A BY $0, B BY $0;
LogicalExpressionSimplifier
This rule contains several types of simplifications.
1) Constant pre-calculation B = FILTER A BY a0 > 5+7; is simplified to B = FILTER A BY a0 > 12; 2) Elimination of negations B = FILTER A BY NOT (NOT(a0 > 5) OR a > 10); is simplified to B = FILTER A BY a0 > 5 AND a <= 10; 3) Elimination of logical implied expression in AND B = FILTER A BY (a0 > 5 AND a0 > 7); is simplified to B = FILTER A BY a0 > 7; 4) Elimination of logical implied expression in OR B = FILTER A BY ((a0 > 5) OR (a0 > 6 AND a1 > 15); is simplified to B = FILTER C BY a0 > 5; 5) Equivalence elimination B = FILTER A BY (a0 v 5 AND a0 > 5); is simplified to B = FILTER A BY a0 > 5; 6) Elimination of complementary expressions in OR B = FILTER A BY (a0 > 5 OR a0 <= 5); is simplified to non-filtering 7) Elimination of naive TRUE expression B = FILTER A BY 1==1; is simplified to non-filtering
MergeForEach
The objective of this rule is to merge together two feach statements, if these preconditions are met:
- The foreach statements are consecutive.
- The first foreach statement does not contain flatten.
- The second foreach is not nested.
-- Original code: A = LOAD 'file.txt' AS (a, b, c); B = FOREACH A GENERATE a+b AS u, c-b AS v; C = FOREACH B GENERATE $0+5, v; -- Optimized code: A = LOAD 'file.txt' AS (a, b, c); C = FOREACH A GENERATE a+b+5, c-b;
OpLimitOptimizer
The objective of this rule is to push the LIMIT operator up the data flow graph (or down the tree for database folks). In addition, for top-k (ORDER BY followed by a LIMIT) the LIMIT is pushed into the ORDER BY.
A = LOAD 'input'; B = ORDER A BY $0; C = LIMIT B 10;
PushUpFilters
The objective of this rule is to push the FILTER operators up the data flow graph. As a result, the number of records that flow through the pipeline is reduced.
A = LOAD 'input'; B = GROUP A BY $0; C = FILTER B BY $0 < 10;
PushDownExplodes
The objective of this rule is to reduce the number of records that flow through the pipeline by moving FOREACH operators with a FLATTEN down the data flow graph. In the example shown below, it would be more efficient to move the foreach after the join to reduce the cost of the join operation.
A = LOAD 'input' AS (a, b, c); B = LOAD 'input2' AS (x, y, z); C = FOREACH A GENERATE FLATTEN($0), B, C; D = JOIN C BY $1, B BY $1;
StreamOptimizer
Optimize when LOAD precedes STREAM and the loader class is the same as the serializer for the stream. Similarly, optimize when STREAM is followed by STORE and the deserializer class is same as the storage class. For both of these cases the optimization is to replace the loader/serializer with BinaryStorage which just moves bytes around and to replace the storer/deserializer with BinaryStorage.
TypeCastInserter
Status: Mandatory
If you specify a schema with the LOAD statement, the optimizer will perform a pre-fix projection of the columns and cast the columns to the appropriate types. An example will illustrate the point. The LOAD statement (a) has a schema associated with it. The optimizer will insert a FOREACH operator that will project columns 0, 1 and 2 and also cast them to chararray, int and float respectively.
A = LOAD 'input' AS (name: chararray, age: int, gpa: float); B = FILER A BY $1 == 1; C = GROUP A By $0;
Pig Properties
The Pig "-propertyfile " option enables you to pass a set of Pig or Hadoop properties to a Pig job. If the value is present in both the property file passed from the command line as well as in default property file bundled into pig.jar, the properties passed from command line take precedence. This property, as well as all other properties defined in Pig, are available to your UDFs via UDFContext.getClientSystemProps()API call (see the Pig UDF Manual.)
You can retrieve a list of all properties using the help properties command.
You can set properties using the set command.
Pig Statistics
Pig Statistics is a framework for collecting and storing script-level statistics for Pig Latin. Characteristics of Pig Latin scripts and the resulting MapReduce jobs are collected while the script is executed. These statistics are then available for Pig users and tools using Pig (such as Oozie) to retrieve after the job is done.
The new Pig statistics and the existing Hadoop statistics can also be accessed via the Hadoop job history file (and job xml file). Piggybank has a HadoopJobHistoryLoader which acts as an example of using Pig itself to query these statistics (the loader can be used as a reference implementation but is NOT supported for production use).
Java API
Several new public classes make it easier for external tools such as Oozie to integrate with Pig statistics.
The Pig statistics are available here: http://pig.apache.org/docs/r0.8.0/api/
The stats classes are in the package: org.apache.pig.tools.pigstats
- PigStats
- JobStats
- OutputStats
- InputStats
The PigRunner class mimics the behavior of the Main class but gives users a statistics object back. Optionally, you can call the API with an implementation of progress listener which will be invoked by Pig runtime during the execution.
package org.apache.pig; public abstract class PigRunner { public static PigStats run(String[] args, PigProgressNotificationListener listener) } public interface PigProgressNotificationListener extends java.util.EventListener { // just before the launch of MR jobs for the script public void LaunchStartedNotification(int numJobsToLaunch); // number of jobs submitted in a batch public void jobsSubmittedNotification(int numJobsSubmitted); // a job is started public void jobStartedNotification(String assignedJobId); // a job is completed successfully public void jobFinishedNotification(JobStats jobStats); // a job is failed public void jobFailedNotification(JobStats jobStats); // a user output is completed successfully public void outputCompletedNotification(OutputStats outputStats); // updates the progress as percentage public void progressUpdatedNotification(int progress); // the script execution is done public void launchCompletedNotification(int numJobsSucceeded); }
Job XML
The following entries are included in job conf:
|
Pig Statistic |
Description |
|
pig.script.id |
The UUID for the script. All jobs spawned by the script have the same script ID. |
|
pig.script |
The base64 encoded script text. |
|
pig.command.line |
The command line used to invoke the script. |
|
pig.hadoop.version |
The Hadoop version installed. |
|
pig.version |
The Pig version used. |
|
pig.input.dirs |
A comma-separated list of input directories for the job. |
|
pig.map.output.dirs |
A comma-separated list of output directories in the map phase of the job. |
|
pig.reduce.output.dirs |
A comma-separated list of output directories in the reduce phase of the job. |
|
pig.parent.jobid |
A comma-separated list of parent job ids. |
|
pig.script.features |
A list of Pig features used in the script. |
|
pig.job.feature |
A list of Pig features used in the job. |
|
pig.alias |
The alias associated with the job. |
Hadoop Job History Loader
The HadoopJobHistoryLoader in Piggybank loads Hadoop job history files and job xml files from file system. For each MapReduce job, the loader produces a tuple with schema (j:map[], m:map[], r:map[]). The first map in the schema contains job-related entries. Here are some of important key names in the map:
|
PIG_SCRIPT_ID CLUSTER QUEUE_NAME JOBID JOBNAME STATUS |
USER HADOOP_VERSION PIG_VERSION PIG_JOB_FEATURE PIG_JOB_ALIAS PIG_JOB_PARENTS |
SUBMIT_TIME LAUNCH_TIME FINISH_TIME TOTAL_MAPS TOTAL_REDUCES |
Examples that use the loader to query Pig statistics are shown below.
Examples
Find scripts that generate more then three MapReduce jobs:
a = load '/mapred/history/done' using HadoopJobHistoryLoader() as (j:map[], m:map[], r:map[]); b = group a by (j#'PIG_SCRIPT_ID', j#'USER', j#'JOBNAME'); c = foreach b generate group.$1, group.$2, COUNT(a); d = filter c by $2 > 3; dump d;
Find the running time of each script (in seconds):
a = load '/mapred/history/done' using HadoopJobHistoryLoader() as (j:map[], m:map[], r:map[]); b = foreach a generate j#'PIG_SCRIPT_ID' as id, j#'USER' as user, j#'JOBNAME' as script_name, (Long) j#'SUBMIT_TIME' as start, (Long) j#'FINISH_TIME' as end; c = group b by (id, user, script_name) d = foreach c generate group.user, group.script_name, (MAX(b.end) - MIN(b.start)/1000; dump d;
Find the number of scripts run by user and queue on a cluster:
a = load '/mapred/history/done' using HadoopJobHistoryLoader() as (j:map[], m:map[], r:map[]); b = foreach a generate j#'PIG_SCRIPT_ID' as id, j#'USER' as user, j#'QUEUE_NAME' as queue; c = group b by (id, user, queue) parallel 10; d = foreach c generate group.user, group.queue, COUNT(b); dump d;
Find scripts that have failed jobs:
a = load '/mapred/history/done' using HadoopJobHistoryLoader() as (j:map[], m:map[], r:map[]); b = foreach a generate (Chararray) j#'STATUS' as status, j#'PIG_SCRIPT_ID' as id, j#'USER' as user, j#'JOBNAME' as script_name, j#'JOBID' as job; c = filter b by status != 'SUCCESS'; dump c;
Find scripts that use only the default parallelism:
a = load '/mapred/history/done' using HadoopJobHistoryLoader() as (j:map[], m:map[], r:map[]); b = foreach a generate j#'PIG_SCRIPT_ID' as id, j#'USER' as user, j#'JOBNAME' as script_name, (Long) r#'NUMBER_REDUCES' as reduces; c = group b by (id, user, script_name) parallel 10; d = foreach c generate group.user, group.script_name, MAX(b.reduces) as max_reduces; e = filter d by max_reduces == 1; dump e;
Specialized Joins
In certain cases, the performance of inner joins and outer joins can be optimized using replicated, skewed, or merge joins.
Replicated Joins
Fragment replicate join is a special type of join that works well if one or more relations are small enough to fit into main memory. In such cases, Pig can perform a very efficient join because all of the hadoop work is done on the map side. In this type of join the large relation is followed by one or more small relations. The small relations must be small enough to fit into main memory; if they don't, the process fails and an error is generated.
Usage
Perform a replicated join with the USING clause (see inner joins and outer joins). In this example, a large relation is joined with two smaller relations. Note that the large relation comes first followed by the smaller relations; and, all small relations together must fit into main memory, otherwise an error is generated.
big = LOAD 'big_data' AS (b1,b2,b3); tiny = LOAD 'tiny_data' AS (t1,t2,t3); mini = LOAD 'mini_data' AS (m1,m2,m3); C = JOIN big BY b1, tiny BY t1, mini BY m1 USING 'replicated';
Conditions
Fragment replicate joins are experimental; we don't have a strong sense of how small the small relation must be to fit into memory. In our tests with a simple query that involves just a JOIN, a relation of up to 100 M can be used if the process overall gets 1 GB of memory. Please share your observations and experience with us.
Skewed Joins
Parallel joins are vulnerable to the presence of skew in the underlying data. If the underlying data is sufficiently skewed, load imbalances will swamp any of the parallelism gains. In order to counteract this problem, skewed join computes a histogram of the key space and uses this data to allocate reducers for a given key. Skewed join does not place a restriction on the size of the input keys. It accomplishes this by splitting the left input on the join predicate and streaming the right input. The left input is sampled to create the histogram.
Skewed join can be used when the underlying data is sufficiently skewed and you need a finer control over the allocation of reducers to counteract the skew. It should also be used when the data associated with a given key is too large to fit in memory.
Usage
Perform a skewed join with the USING clause (see inner joins and outer joins).
big = LOAD 'big_data' AS (b1,b2,b3); massive = LOAD 'massive_data' AS (m1,m2,m3); C = JOIN big BY b1, massive BY m1 USING 'skewed';
Conditions
Skewed join will only work under these conditions:
- Skewed join works with two-table inner join. Currently we do not support more than two tables for skewed join. Specifying three-way (or more) joins will fail validation. For such joins, we rely on you to break them up into two-way joins.
- The pig.skewedjoin.reduce.memusage Java parameter specifies the fraction of heap available for the reducer to perform the join. A low fraction forces pig to use more reducers but increases copying cost. We have seen good performance when we set this value in the range 0.1 - 0.4. However, note that this is hardly an accurate range. Its value depends on the amount of heap available for the operation, the number of columns in the input and the skew. An appropriate value is best obtained by conducting experiments to achieve a good performance. The default value is 0.5.
- Skewed join does not address (balance) uneven data distribution across reducers. However, in most cases, skewed join ensures that the join will finish (however slowly) rather than fail.
Merge Joins
Often user data is stored such that both inputs are already sorted on the join key. In this case, it is possible to join the data in the map phase of a MapReduce job. This provides a significant performance improvement compared to passing all of the data through unneeded sort and shuffle phases.
Pig has implemented a merge join algorithm, or sort-merge join, although in this case the sort is already assumed to have been done (see the Conditions, below). Pig implements the merge join algorithm by selecting the left input of the join to be the input file for the map phase, and the right input of the join to be the side file. It then samples records from the right input to build an index that contains, for each sampled record, the key(s) the filename and the offset into the file the record begins at. This sampling is done in the first MapReduce job. A second MapReduce job is then initiated, with the left input as its input. Each map uses the index to seek to the appropriate record in the right input and begin doing the join.
Usage
Perform a merge join with the USING clause (see inner joins and outer joins).
C = JOIN A BY a1, B BY b1, C BY c1 USING 'merge';
Conditions
Condition A
Inner merge join (between two tables) will only work under these conditions:
- Between the load of the sorted input and the merge join statement there can only be filter statements and
foreach statement where the foreach statement should meet the following conditions:
- There should be no UDFs in the foreach statement.
- The foreach statement should not change the position of the join keys.
- There should be no transformation on the join keys which will change the sort order.
- Data must be sorted on join keys in ascending (ASC) order on both sides.
- Right-side loader must implement either the {OrderedLoadFunc} interface or {IndexableLoadFunc} interface.
- Type information must be provided for the join key in the schema.
The Zebra and PigStorage loaders satisfy all of these conditions.
Condition B
Outer merge join (between two tables) and inner merge join (between three or more tables) will only work under these conditions:
- No other operations can be done between the load and join statements.
- Data must be sorted on join keys in ascending (ASC) order on both sides.
- Left-most loader must implement {CollectableLoader} interface as well as {OrderedLoadFunc}.
- All other loaders must implement {IndexableLoadFunc}.
- Type information must be provided for the join key in the schema.
The Zebra loader satisfies all of these conditions.
An example of a left outer merge join using the Zebra loader:
A = load 'data1' using org.apache.hadoop.zebra.pig.TableLoader('id:int', 'sorted'); B = load 'data2' using org.apache.hadoop.zebra.pig.TableLoader('id:int', 'sorted'); C = join A by id left, B by id using 'merge';
Both Conditions
For optimal performance, each part file of the left (sorted) input of the join should have a size of at least 1 hdfs block size (for example if the hdfs block size is 128 MB, each part file should be less than 128 MB). If the total input size (including all part files) is greater than blocksize, then the part files should be uniform in size (without large skews in sizes). The main idea is to eliminate skew in the amount of input the final map job performing the merge-join will process.
Zebra Integration
For information about how to integrate Zebra with your Pig scripts, see Zebra and Pig.


If you want to send yor child to our Camp...
Fill this form
Fill this form
Privacy Policy: We will not share any of your information with anyone else.
Privacy Policy: We will not share any of your information with anyone else.


Constructing Cultural Diversity Measures
The cultural diversity domains were informed by stakeholders meetings, workshops, available data sources, and existing literature on the capital. The cultural domains, religion and ancestry, are summarized using the following two measures.
1. Richness: The richness measure is the number of groups within the domain, in this case the number of religious groups or the number of different ancestries.
2. Diversity: Diversity is a function of the number of groups and group members. The index will approach zero when the group members are concentrated in a few groups and will approach one when the group members are evenly distributed. The Gini-Simpson Diversity Index is calculated as:
where R is the richness (number of groups) and p is the relative frequency, the number of group members divided by the total number of members.
Interpreting Domain Measures
Choropleths display the value of the measures at the county level for the selected state. The boxplots below the choropleth display the county and city measures for each state. The filled circles over the box plot indicate the relative rurality of the county or city; the more saturated the shade of orange, the more rural, and the more saturated the shade of olive, the more urban a county or city is.
Index of Relative Rurality
We display variable boxplots by county relative rurality. We assign county rurality using Waldorf and Kim's 2010 Index of Relative Rurality (IRR). The IRR is based on population size, density, remoteness, and built-up area, and ranges from 0 (most urban) to 1 (most rural). Read more about the index here.


This sample registration form uses various types of input elements and buttons.
For more info about individual controls, including new ways they can be styled, see the Everyday Widgets sample.


We all have various sides of our self, and each of them has their own thoughts and stories about experiences. And though there are hundreds of dimensions to it, there are some we identify ourselves with more than others, some that feel more us to us than others.
When I think about how the pandemic has affected me, I can’t think of one absolute way it has. The experience hasn’t been binary, one that can be categorized into good or bad. We all have various sides of our self, and each of them has their own thoughts and stories about experiences. And though there are hundreds of dimensions to it, there are some we identify ourselves with more than others, some that feel more us to us than others.
I’ve tried to think about what those are for me, bring them to light, and get them to talk. Most of them have adapted quite well to the changes necessitated by the constraints of the situation.
The different selfs
Since we spend a better part of our lives doing our day job, it naturally defines us in a significant way, if not completely. I work as a software engineer, and it’s one of my important identities, it’s also the one that’s most visible or known to the outside world.
One of the integral parts of my job, an activity that is also close to my heart, is to always keep learning. That’s another thing I identify myself with - as someone eager to learn and open to change.
A fundamental, and in some ways more important, part of my identity is one that’s obsessed with practising a healthy lifestyle. I say important because I believe it nourishes the other selfs, and fundamental because it’s less fluid in nature than the others.
One other important dimension is the part of me that cares deeply about animals. I say cares and not loves, because that’s an important distinction, and even though I love some animals, I want to be mindful about the difference.
There’s one other aspect of me has stood the test of time. I don’t have much control over it, and I’ve had a love-hate relationship with it, more hate than love - my introverted nature. More hate than love because I’ve only recently started to accept, and nurture it, rather than treating it like a problem that needed fixing.
The next one is a close kin of the self that wants to lead a healthy life, but is more opinionated on how to go about it. It believes in spending time in nature, and doing physically challenging things to keep the body and the mind in good shape. The curious and learning part of me constantly feeds this self, with new ideas, challenges to tackle and goals to pursue.
The last one I can think of is the most volatile of them all, and has only come to consciousness very recently. It swims up to the surface and goes back to hiding depending on the things that’s happening around me - the one that thinks of itself as an immigrant in a foreign country, a guest in someone else’s place.
Here I try to prod each one of them to express their thoughts about how they feel being sheltered in place.
As a software engineer,
it’s fairly easy to work remotely and still be productive. While I miss the walks and talks around the office with my peers to get coffee during the day, I’ve made up for it by experimenting with different kinds of plant-based milk coffees at home. After several weeks of trial-and-error, it’s getting close to my liking.
I hate that wifi at home is not as good and causes, part frustrating, part funny, technical issues while talking to my colleagues on video calls.
I also miss the desk setup and multiple monitors I used to have in the office, but for now, I’m managing with an REI camp chair, a laptop and a monitor. It’s easy to get lost in work and forget how or where I’m sitting at home. Stiffness from bad body postures used to be a problem in the first weeks, but I’ve gotten more mindful about that over time.
I like that I don’t have to wait for meeting rooms to free up to chat with people - they’re just a few clicks and a video call away. I think it’s more productive and collaborative than an open-office setting.
I like that I don’t have to rush to catch the train and sit through a long commute to get to work in the morning, and that I can start working early in the day when my mind is at its best.
In the first few weeks, I worked more than usual, and so did my peers. But I’ve gotten better at drawing boundaries between work and personal life, even without a physical one separating the two.
One other thing I don’t miss is the incessant chatter in the office, where unnecessary small talk is confused with collaboration and coding is confused with mindless typing.
Given the things I don’t miss from going to work and the things I love about working from home, I’d do more remote work going forward for sure, and hope companies embrace it more too.
As an avid reader,
I like that the two or more hours I used to spend riding in train to go to work, I now spend riding in train of thoughts, one that takes me to farther and better places. Reading not only helps me unplug from the stress and concerns of work, but the comfort and peace of doing it from home improves my mood. The absence of distractions of a typical workday has helped me do more of a fewer things rather than juggle a little bit of everything. Long stretches of undistracted time allows me to focus on reading books, long-form articles and write more, rather than filling the tiny slices of time over the day with news and junk.
The absence or lack of something usually makes you realize its importance, but it’s interesting in a way that having more, and not less, time in hand has made me realize how much of it was being taken for granted.
I’m also starting to see the benefits of consistency and discipline, of cultivating enough patience to do something everyday despite the initial aversion and resistance, and how it can make a big difference relatively quickly. Though I have been trying to get into the habit of writing for over a year, I’ve only been this consistent and serious about it since I started working from home. The world had to hit brakes on its operations and get rid of distractions for me to start writing. It’s like the Universe really wanted to teach me good habits.
As an outdoor enthusiast,
I was really worried about not being able to travel. I had a few races that I had signed up for and a few trips I had planned for this year. But sitting at home, not having hiked or biked in a couple of months now has taught me that I don’t need to go to new places to freshen up my mind, or to enjoy outdoors every time I head out. That I can just step out of the door and start running until I get tired and run back. That it can be that simple.
I started running a couple of years ago for the fun of it, and maybe a little to lose weight too, but over time, I started getting obsessed over quantifying the benefits of every activity - the calories burnt, pounds lost, and speed etc,. I’m now back to those simple days. Not every run now goes in Strava, not every race has to end up in PRs, not every walk now is a hike and not every bike ride is captured in photos.
The solace I used to find in the outdoors, I’ve found it in reading and writing too. Although there is something magical about being out in nature, I realize that one of the reasons it felt so good was the time it gave me to pause, and reflect on things. Turns out books do the same.
As an introvert,
I’m in love with the silence. It not only helps me think clearly, and be more productive at work, it also puts me in a much better mood overall. Even after working long hours, I find that I have the time and energy to go on walks with my wife, cook, go for a run, read, write and talk to friends and family on the phone.
I like the idea of minimizing sounds that fall in my ears, the idea of letting them in willingly rather than being bombarded with unnecessary and unsolicited noises all day long and being forcefully made a passive member in every conversation around me.
As an immigrant
from India, I see the contrast in how people from the two countries are reacting to the pandemic, both common people and those in power. Being from the majority group in India, and a part of one of the minority groups(immigrants) here in the US, it gives some perspectives on how stereotypes and misinformation against minorities are used for political and capital gains.
While I’m concerned about the well-being of older members of my family back home and wish I were there, a part of me feels relieved that I’m not with them to inadvertently infect them.
I see through the lies and the facade of developed societies and feel hopeful and optimistic, to see the potential of the developing nations and their simple, yet effective ways of doing things.
Looking at the things that have worked for countries and things that haven’t, I realize that more technology is not necessarily the right solution to all problems. Common sense and empirical evidence are perfectly fine at times.
As an animal rights believer,
I love that non-humans have started to come out of hiding and are doing so without the fear of us harming, or killing them, that at least for a little while they regain their rights over the resources of this planet, and not everything they consume are left-overs from humans.
I’m terribly sad and shocked by the fate of hundreds of thousands, if not millions, of animals in factory farms that were killed because humans couldn’t afford to raise and kill them to eat. I can’t decide which is worse, simply killing them, or killing to eat them.
I like that meat substitutes are seeing tremendous growth and hope that the trend continues long after our lives get back to normal. Although it’s a result of shortage of meat in markets right now, I hope that reduced meat and dairy consumption becomes the new normal, and people see it for what it actually is, and the problems it puts us into.
Slow life vs fast life
We say the world has slowed down during the pandemic, but what does it mean to be slow?
Have we truly slowed now, or were we unnecessarily fast before?
What was the rush anyway, and where were we all headed?
Were we navigating towards a perceived sense of growth and success, or were we merely racing towards a mountain of problems created by our own greed and speed?
And if the rush was in fact necessary, why does slow feel better?
If this is the age of mind and innovation, than of body and physical labor, then why were cruising in autopilot, without stopping to think about the destination?
Isn’t it ironic that the fast life we chose in pursuit of something good only made us more miserable and unhealthy, and this life of nothingness is making us happier, and healthier?
Isn’t it ironic that we’re regaining the connections we had lost, by distancing ourselves? That we had to be physically social-distanced to close the mental and emotional gap between each other?
Recommended Reading
Ideas and learnings from a lot of reading and collaboration between multiple dimensions of my self has gone into producing this. Of all the news and information that’s out there about the pandemic, a majority of them either portray humans as the victims, not the villains, or blame a specific section of people for all the problems. With such lopsided, and biased opinions floating around, this one below stands out with a different perspective on things and makes for a wonderful read.


In the constructor, it computes the mean and the standard deviation over all the frames in the given DataSet (by default, only for inputs).
In the constructor, it computes the mean and the standard deviation over all the frames in the given DataSet (by default, only for inputs). Then, when calling pre-processing methods, it normalizes each column by this computed mean and stdv. (substracts the mean, then divides by the standard deviation).As a result, the mean of the full set of frames given by the concatenation of all calls to setExample() will be 0, and the variance will be 1.
Alphabetic index HTML hierarchy of classes or Java


{% extends "base.html" %} {% block app_content %}
{{ user.username }}Saved groups: {{ groups.count() }} |
You have not saved any groups yet!
{% else %}| Title | Individuals | Created | ||
|---|---|---|---|---|
| {{ group.title }} | {{ group.indiv_display }} | {{ moment(group.creation_time).calendar() }} | Load | Delete |


Contains classes pertaining to the parsing of Boolean expressions.
As of now, only contains one class: BooleanParser. This class provides one public method: parse(String expression), which parses a String representing a Boolean expression and returns a Boolean value.


# File lib/rack/handler.rb, line 46 def self.default(options = {}) # Guess. if ENV.include?("PHP_FCGI_CHILDREN") # We already speak FastCGI options.delete :File options.delete :Port Rack::Handler::FastCGI elsif ENV.include?("REQUEST_METHOD") Rack::Handler::CGI else pick ['thin', 'puma', 'webrick'] end end


{% if direct_trades or sell_trades or buy_trades %}
| You | want to sell | {{t.share.runner}} | to {{t.buyer}} | {% else %}{{t.seller}} | wants to sell | {{t.share.runner}} | to you | {% endif %} {% else %} {% if t.buyer == user.investor %}You | want to buy | {{t.share.runner}} | from {{t.seller}} | {% else %}{{t.buyer}} | wants to buy | {{t.share.runner}} | from you | {% endif %} {% endif %}for {{t.print_price}} | {% if t.creator == user.investor %} {% else %} {% endif %} |
| You | want to sell | {{t.share.runner}} | to anyone | for {{t.print_price}} | |||||||||||||
| You | want to buy | {{t.runner}} | from anyone | for {{t.print_price}} |


_Automated Pipelines_
_Automatically assemble uploads_
_RUN_SISTR_
_WRITE_BACK_TO_SAMPLE_METADATA_
Coverage
_Edit_
_cancel_
_save_
_Required Coverage_
N/A
_Required Coverage_
N/A
_Genome Size_
N/A
_Min Coverage_
x
_Max Coverage_
x
_Genome Size_
bp


THE FIFTH ELEMENT A film review by James Berardinelli Copyright 1997 James Berardinelli
RATING (0 TO 10): 5.5 Alternative Scale: ** out of ****
United States, 1997 U.S. Release Date: 5/9/97 (wide) Running Length: 2:07 MPAA Classification: PG-13 (Brief nudity, violence, profanity) Theatrical Aspect Ratio: 2.35:1
Cast: Bruce Willis, Milla Jovovich, Gary Oldman, Ian Holm, Chris Tucker, Luke Perry Director: Luc Besson Producer: Patrice Ledoux Screenplay: Luc Besson & Robert Mark Kamen Cinematography: Thierry Arbogast Music: Eric Serra U.S. Distributor: Columbia Pictures
THE FIFTH ELEMENT uses one of the newest tricks in the science fiction motion picture handbook: perform a visual and aural assault on viewers in the vain hope that they won't notice the lack of substance, logic, and intelligence. For a few recent examples of where this tactic has been employed, check out STARGATE, INDEPENDENCE DAY, and JUDGE DREDD. THE FIFTH ELEMENT, the summer of 1997's first big-budget disappointment, is right at home in such company. A lot of money was spent on this film, but $100 million doesn't guarantee a good product. Maybe someone should have thought of spending a few more dollars on a better script.
Going into the film, I was guardedly optimistic. As a result of what I'd seen in the two-minute theatrical trailer, visions of BLADE RUNNER danced in my head (I guarantee that no one in their right mind would make such a ludicrous association after seeing THE FIFTH ELEMENT). And the director, Luc Besson, is a well-respected artist whose short- but-impressive resume includes such international hits as LA FEMME NIKITA and THE PROFESSIONAL (LEON). I figured that if anyone could handle the task of putting together a rousing science fiction adventure with Bruce Willis in the lead, it was Besson. Alas, such is not the case.
What can I say about the storyline? There's not much there, and what there is, doesn't make a lot of sense. If you see THE FIFTH ELEMENT, do not, under any circumstances, attempt to reason out what's going on. In general, science fiction tales play fast and loose with reality, but here, it's so flagrant that even a moment's intelligent consideration of what's on the screen will shatter any hope of suspending disbelief.
The goal in THE FIFTH ELEMENT is to save Earth from a gargantuan evil that's hovering somewhere beyond the fringes of the solar system, blasting space ships as it waits around for forty-eight hours so our heroes can destroy it before it turns the planet into a cinder. Nothing can stop this nearly-omnipotent enemy (which looks like the Genesis planet from the STAR TREK movies) except The Fifth Element -- the most powerful weapon ever devised. Things aren't straightforward, however, because the Fifth Element is actually a girl who can't speak English (Milla Jovovich), and she's missing four key stones without which she can't do her job.  The quest, which starts on Earth during the twenty-third century, takes us to an outer space vacation resort where the stones are supposedly hidden.
THE FIFTH ELEMENT is populated by all manner of flashed, bangs, and booms. New York City, circa 2200, has been rendered with eye-popping splendor by the effects wizards at Digital Domain. (Check out rush hour in the future if you think it's bad now!) Bright colors -- reds, oranges, and yellows -- are in evidence everywhere, from Bruce Willis' outfit to Milla Jovovich's hair. And Eric Serra's throbbing, cacophonous score tests the limits of theater speakers.
When it comes time to discuss the rest of the movie, however, there's not much to talk about. When a wisecracking actor like Bruce Willis and a hammy performer like Gary Oldman are reduced to ineffectual bystanders, there's a problem. Willis and Oldman don't have much to do, and, when they're on screen, they fade into the background, dwarfed by the special effects. Willis, whose John McLane is the DIE HARD series, is used to taking center stage, but he can't compete with the visual busy-ness of THE FIFTH ELEMENT, and that makes him a pretty dull hero. Meanwhile, with her fiery hair and occasionally-revealing costumes, Jovovich makes an impression, although her effectiveness has little to do with acting and less to do with dialogue (she doesn't have much to say). Only Chris Tucker, who plays an insanely energetic, over-the-top talk show host (a loquacious Dennis Rodman in outer space) manages to hold his own amidst the digital chaos, but his outrageousness starts to wear thin after about two minutes.
I will give Besson some credit. He never takes things too seriously, which is a good idea, because a somber version of THE FIFTH ELEMENT would have made it a candidate for the year's Bottom 10 list. The director, working from a comic book-type script that he co-wrote with Robert Mark Kamen (and which bears a certain resemblance to one of the segments of HEAVY METAL), injects a ton of offbeat, sporadically- effective humor into the movie, frequently causing THE FIFTH ELEMENT to cross over the line into self-parody. This unique approach kills any possible sense of adventure or suspense (the numerous action scenes are obligatory and perfunctory, and never remotely exciting), but it at least offers something unusual.
Once THE FIFTH ELEMENT's assault on the senses starts to wear off, it's easy to become restless. After all, in keeping with the "epic" sweep demanded by such an obese budget, the film had to clock in at over two hours, even though there's no excuse for a movie with so little going on to last for so long. At least the payoff is worthwhile in a campy sense. Where else can you see sweat rung from a handkerchief become integral to saving the world? Besson may have misfired with THE FIFTH ELEMENT, but at least he does it with flair and a sense of humor.
- James Berardinelli e-mail<EMAIL_ADDRESS>ReelViews web site: http://www.cybernex.net/~berardin
The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


Our People
Meet the people behind DIVE.IS
Tania Sofia Baia Roque
Popcorn
I am Portuguese but relocated to Iceland about five years ago. I fell in love with diving here in Iceland. For the last three years, however, I have been going between Iceland and Thailand for a perfect mix of diving locations. In addition to all water sports, I love traveling, reading, yoga, animals, and music.
Be kindSiobhan Diana White
Siobhan
I have been diving since 1999 and became both a BSAC and PADI instructor in 2008/9 in the murky waters of the UK. I have a BSc in Hospitality and Tourism Management and before joining the DIVE.IS team in summer 2011 I ran a marine conservation project in the Bahamas. I am passionate marine conservationist as i love the variety of life that can be found in our oceans, and feel like we should be doing everything in our power to protect it!
Courage is not the absence of fear, But rather the judgement that there is something more important than fear. The brave may not live for ever, but the cautious do not live at all.Scott Fitzsimmons
don't call me that
I was born and raised in Canada but have been hiding from the cold for the last 5 years on sailboats in Australia and luxury resorts in Oman I have decided to go back to a climate where shorts don't exist. I am happiest either in the mountains or in the water and finally found a place with easy access to both. Just don't ask me to pronounce anything in Icelandic eh.
What about second breakfast?Rakel Magnúsdóttir
Raxpal
This tiny viking was born and raised in Iceland.  No matter the task she gets it done. Home sweet home kind of girl but also has a passion for traveling the world, yoga/yoga instructor, scuba diving, hiking and snowboarding.
You have to trust in something -- your gut, destiny, life, karma, whatever. This approach has never let me down, and it has made all the difference in my life." -- Steve JobsKolbrún ýr Gunnarsdóttir
Kolbrún
You make your own happiness!Our Philosophy
Getting a bit better every day
DIVE.IS – Mission Statement
Our aim is to fulfil the highest standards with regard to dive education, dive safety, equipment, environmentally friendly actions and enlightment for the best possible prices, leading to customer and employee satisfaction beyond their expectations.
DIVE.IS – Vision
Our vision is to remain the leading dive center in Iceland and strive to improve in everything we do every single day. We want to keep on setting the standards for recreational diving and snorkeling in Iceland as we have done since 1997. Our intention is to remain an intimate business with the maximum of personal and professional service and commitment to our customers and employees.
DIVE.IS – Environmental Responsibility
We believe know that we can make a difference. As divers we have the privilege to admire the wonders of nature close up on a day to day basis. Looking at how all organisms on this planet work and live together in a symbiosis assures us of being part of something big, something perfect & divine. We know we have to do our part in protecting earth as an organism and all its living things … and we do it gladly and we are proud of it:
- We try to minimise our carbon footprint by using vehicles with a low rate of consumption
- We have modified some of our vehicles to methane
- We try to refrain from printing paper as much as possible
- We recycle our waste
- We give preference to environmentally safe products and local services
- We are a member and supporter Project Aware, an environmental organisation specialising on the protection of our oceans
- We organise and conduct clean-ups at different dive sites around Iceland
- We educate our staff about sustainability
- We educate our students and clients about the importance of protecting nature and how to do it on their stay in Iceland
Funfacts
Funfacts about DIVE.IS
Water Temperature
2-4° - Water temperature: just above freezing Visibility: Depends solely on the quality of your dive mask. Where: In a lava fissure ripped open by earthquakes What else: Combine it with other great nature sights and activities. Why: Because you want to take this memory home.
Did you know?
You can drink Silfra.
Total Guides?
35.
5-Star Reviews?
over 2000 & over 1000 in 2016
How are we rated on Trip Advisor?
No. 1 of all 50 water activities in Iceland


This was meant as your personal summary of what we have done till now. One possible solution is the illustration made by Julie, also at the end of every chapter of Taylor's textbook, you will find excellent summaries of what we have covered till now.
Which of the following force are conservative? All three forces depend only on \( \boldsymbol{r} \) and satisfy the first condition for being conservative.
The curl is zero and the force is conservative. The potential energy is upon integration \( V(x)=-k(1/2x^2+y^2+3/2z^2) \). Taking the derivative shows that this is indeed the case since it gives back the force.
This force is also conservative since it depends only on the coordinates and its curl is zero. To find the potential energy, since the integral is path independent, we can choose to integrate along any direction. The simplest is start from \( x=0 \) as origin and follow a path along the \( x \)-axis (which gives zero) and then parallel to the \( y \)-axis, which results in \( V(x,y) = -xy \). Taking the derivative with respect to \( x \) and \( y \) gives us back the expression for the force.
Here the curl is \( (0,0,2) \) and the force is not conservative.
See the answers to each exercise above.
The Lennard-Jones potential is often used to describe the interaction between two atoms or ions or molecules. If you end up doing materals science and molecular dynamics calculations, it is very likely that you will encounter this potential model. The expression for the potential energy is
$$ V(r) = V_0\left((\frac{a}{r})^{12}-(\frac{b}{r})^{6}\right), $$where \( V_0 \), \( a \) and \( b \) are constants and the potential depends only on the relative distance between two objects \( i \) and \( j \), that is \( r=\vert\vert\boldsymbol{r}_i-\boldsymbol{r}_j\vert\vert=\sqrt{(x_i-x_j)^2+(y_i-y_j)^2+(z_i-z_j)^2} \).
The following Python code plots the potential
# Common imports
import numpy as np
from math import *
import matplotlib.pyplot as plt
Deltar = 0.01
#set up arrays
rinitial = 1.8
rfinal = 3.
n = ceil((rfinal-rinitial)/Deltar)
r = np.zeros(n)
for i in range(n):
r[i] = rinitial+i*Deltar
V = np.zeros(n)
# Initial conditions as compact 2-dimensional arrays
a = 2.0
b = 2.0
V0 = 10.0
V = V0*((a/r)**(12)-(b/r)**6)
# Plot position as function of time
fig, ax = plt.subplots()
#ax.set_xlim(0, tfinal)
ax.set_ylabel('V')
ax.set_xlabel('r')
ax.plot(r, V)
fig.tight_layout()
plt.show()
Here there is only one equilibrium point when we take the derivative of the potential with respect to the relative distance.
The derivative with respect to \( r \), the relative distance, is
$$ \frac{dV}{dr} = -6V_0\left(2\frac{a^{12}}{r^{13}}-\frac{b^6}{r^7}\right), $$and this is zero when
$$ r = 2^{1/6}\frac{a^2}{b}. $$If we choose \( a=2 \) and \( b=2 \) then \( r=2\times 2^{1/6} \). Since the second derivative is positive for all \( r \) for our choices of \( a \) and \( b \) (convince yourself about this), then this value of \( r \) has to correspond to a minimum of the potential. This agrees with our graph from the figure above (run the code to produce the figure).
From the previous exercise we have
$$ \frac{dV}{dr} = -6V_0\left(2\frac{a^{12}}{r^{13}}-\frac{b^6}{r^7}\right). $$We need the gradient and since the force on particle \( i \) is given by \( \boldsymbol{F}_i=\boldsymbol{\nabla}_i V(\boldsymbol{r}_i-\boldsymbol{r}_j) \), we obtain
$$ \boldsymbol{F}_i=6V_0\left(2(\frac{a}{\vert\vert\boldsymbol{r}_i-\boldsymbol{r}_j\vert\vert})^{12}-(\frac{b}{\vert\vert\boldsymbol{r}_i-\boldsymbol{r}_j\vert\vert})^6\right)\frac{\boldsymbol{r}_i-\boldsymbol{r}_j}{\vert\vert\boldsymbol{r}_i-\boldsymbol{r}_j\vert\vert^2}. $$Here \( r = \vert\vert \boldsymbol{r}_i-\boldsymbol{r}_j\vert \vert \). If we have more than two particles, we need to sum over all other particles \( j \). We have thus to introduce a sum over all particles \( N \). The force on particle \( i \) at position \( \boldsymbol{r}_i \) from all particles \( j \) at their positions \( \boldsymbol{r}_j \) results in the equation of motion (note that we have divided by the mass \( m \) here)
$$ \boldsymbol{a}_i=\frac{d^2\boldsymbol{r}_i}{dt^2} = \frac{6V_0}{m} \sum_{j \neq i}^{N}\left(2(\frac{a}{\vert\vert\boldsymbol{r}_i-\boldsymbol{r}_j\vert\vert})^{12}-(\frac{b}{\vert\vert\boldsymbol{r}_i-\boldsymbol{r}_j\vert\vert})^6\right)\frac{\boldsymbol{r}_i-\boldsymbol{r}_j}{\vert\vert\boldsymbol{r}_i-\boldsymbol{r}_j\vert\vert^2}. $$This is also a conservative force, with zero curl as well.
Relevant reading here is Taylor chapter 5 and the lecture notes on oscillations. Similalry, Taylor section 4.8 and Malthe-Sørenssen section 11.3 on the potential surfaces are of relevance here.
Consider a particle of mass \( m \) moving in a one-dimensional potential,
$$ V(x)=-\alpha\frac{x^2}{2}+\beta\frac{x^4}{4}. $$The following Python code gives a plot of potential
# Common imports
import numpy as np
from math import *
import matplotlib.pyplot as plt
Deltax = 0.01
#set up arrays
xinitial = -2.0
xfinal = 2.0
n = ceil((xfinal-xinitial)/Deltax)
x = np.zeros(n)
for i in range(n):
x[i] = xinitial+i*Deltax
V = np.zeros(n)
# Initial conditions as compact 2-dimensional arrays
alpha = 0.81
beta = 0.5
print(sqrt(alpha/beta))
V = -alpha*x*x*0.5 + beta*(x**4)*0.25
# Plot position as function of time
fig, ax = plt.subplots()
#ax.set_xlim(0, tfinal)
ax.set_xlabel('x')
ax.set_ylabel('V[s]')
ax.plot(x, V)
fig.tight_layout()
plt.show()
Here we have chosen \( \alpha=0.81 \) and \( \beta=0.5 \). Taking the derivative of \( V \) with respect to \( x \) gives two minima (and it is easy to see here that the second derivative is positive) at \( x\pm\sqrt{\alpha/\beta} \) and a maximum at \( x=0 \). The derivative is
$$ \frac{dV}{dx} = -\alpha x + \beta x^3, $$which gives when we require that it should equal zero the above values. As we can see from the plot (run the above Python code), we have two so-called stable equilibrium points (where the potential has its minima) and an unstable equilibrium point.
The force is conservative since it depends only on \( x \) and has a curl which is zero.
In the solution to the previous exercise we listed the values where the derivatives of the potential are zero. Taking the second derivatives we have that
$$ \frac{d^2V}{dx^2} = -\alpha + 3\beta x^2, $$and for \( \alpha,\beta > 0 \) (we assume they are positive constants) we see that when \( x=0 \) that the the second derivative is negative, which means this is a maximum. For \( x=\pm\sqrt{\alpha/\beta} \) we see that the second derivative is positive. Thus these points correspond to two minima.
Assume now we Taylor-expand the potential around one of these minima, say \( x_{\mathrm{min}}=\sqrt{\alpha/\beta} \). We have thus
$$ V(x) = V(x_{\mathrm{min}})+(x-x_{\mathrm{min}})\frac{dV}{dx}\vert_{x_{\mathrm{min}}}+\frac{1}{2}(x-x_{\mathrm{min}})^2\frac{d^2V}{dx^2}\vert_{x_{\mathrm{min}}}+\dots $$Since we are at point where the first derivative is zero and inserting the value for the second derivative of \( V \), keeping only terms up to the second derivative and finally taking the derivative with respect to \( x \), we find the expression for the force
$$ F(x) = -(x-x_{\mathrm{min}})\frac{d^2V}{dx^2}\vert_{x_{\mathrm{min}}}, $$and setting in the expression for the second derivative at the minimum we find
$$ F(x) = -2\alpha(x-\sqrt{\frac{\alpha}{\beta}}). $$Thus our effective spring constant \( k=2\alpha \).
Here we simplify our force by rescaling our zeroth point so that we have a force (setting \( x_{\mathrm{min}}=0 \))
$$ F(x) = -kx, $$with \( k=2\alpha \). Defining a natural frequency \( \omega_0 = \sqrt{k/m} \), where \( m \) is the mass of our particle, we have the following equation of motion
$$ \frac{d^2x}{dt^2}=-\omega_0^2x, $$which has as analytical solution \( x(t)=A\cos{(\omega_0t)}+B\sin{(\omega_0t)} \) and velocity \( x(t)=-\omega_0A\sin{(\omega_0t)}+\omega_0B\cos{(\omega_0t)} \). The initial conditions are used to define \( A \) and \( B \).
The code here implements Euler's method for the Earth-Sun system using a more compact way of representing the vectors. Alternatively, you could have spelled out all the variables \( v_x \), \( v_y \), \( x \) and \( y \) as one-dimensional arrays. It tests conservation of potential and kinetic energy as functions of time, in addition to the total energy, again as function of time
Note: in all codes we have used scaled equations so that the gravitational constant times the mass of the sum is given by \( 4\pi^2 \) and the mass of the earth is set to one in the calculations of kinetic and potential energies. Else, we would get very large results.
# Common imports
import numpy as np
import pandas as pd
from math import *
import matplotlib.pyplot as plt
import os
# Where to save the figures and data files
PROJECT_ROOT_DIR = "Results"
FIGURE_ID = "Results/FigureFiles"
DATA_ID = "DataFiles/"
if not os.path.exists(PROJECT_ROOT_DIR):
os.mkdir(PROJECT_ROOT_DIR)
if not os.path.exists(FIGURE_ID):
os.makedirs(FIGURE_ID)
if not os.path.exists(DATA_ID):
os.makedirs(DATA_ID)
def image_path(fig_id):
return os.path.join(FIGURE_ID, fig_id)
def data_path(dat_id):
return os.path.join(DATA_ID, dat_id)
def save_fig(fig_id):
plt.savefig(image_path(fig_id) + ".png", format='png')
# Initial values, time step, positions and velocites
DeltaT = 0.0001
#set up arrays
tfinal = 100 # in years
n = ceil(tfinal/DeltaT)
# set up arrays for t, a, v, and x
t = np.zeros(n)
v = np.zeros((n,2))
r = np.zeros((n,2))
# setting up the kinetic, potential and total energy, note only functions of time
EKinetic = np.zeros(n)
EPotential = np.zeros(n)
ETotal = np.zeros(n)
# Initial conditions as compact 2-dimensional arrays
r0 = np.array([1.0,0.0])
v0 = np.array([0.0,2*pi])
r[0] = r0
v[0] = v0
Fourpi2 = 4*pi*pi
# Setting up variables for the calculation of energies
# distance that defines rabs in potential energy
rabs0 = sqrt(sum(r[0]*r[0]))
# Initial kinetic energy. Note that we skip the mass of the Earth here, that is MassEarth=1 in all codes
EKinetic[0] = 0.5*sum(v0*v0)
# Initial potential energy (note negative sign, why?)
EPotential[0] = -4*pi*pi/rabs0
# Initial total energy
ETotal[0] = EPotential[0]+EKinetic[0]
# Start integrating using Euler's method
for i in range(n-1):
# Set up the acceleration
# Here you could have defined your own function for this
rabs = sqrt(sum(r[i]*r[i]))
a = -Fourpi2*r[i]/(rabs**3)
# update Energies, velocity, time and position using Euler's forward method
v[i+1] = v[i] + DeltaT*a
r[i+1] = r[i] + DeltaT*v[i]
t[i+1] = t[i] + DeltaT
EKinetic[i+1] = 0.5*sum(v[i+1]*v[i+1])
EPotential[i+1] = -4*pi*pi/sqrt(sum(r[i+1]*r[i+1]))
ETotal[i+1] = EPotential[i+1]+EKinetic[i+1]
# Plot energies as functions of time
fig, axs = plt.subplots(3, 1)
axs[0].plot(t, EKinetic)
axs[0].set_xlim(0, tfinal)
axs[0].set_ylabel('Kinetic energy')
axs[1].plot(t, EPotential)
axs[1].set_ylabel('Potential Energy')
axs[2].plot(t, ETotal)
axs[2].set_xlabel('Time [yr]')
axs[2].set_ylabel('Total Energy')
fig.tight_layout()
save_fig("EarthSunEuler")
plt.show()
We see very clearly that Euler's method does not conserve energy!! Try to reduce the time step \( \Delta t \). What do you see?
With the Euler-Cromer method, the only thing we need is to update the position at a time \( t+1 \) with the update velocity from the same time. Thus, the change in the code is extremely simply, and energy is suddenly conserved. Note that the error runs like \( O(\Delta t) \) and this is why we see the larger oscillations. But within this oscillating energy envelope, we see that the energies swing between a max and a min value and never exceed these values.
# Common imports
import numpy as np
import pandas as pd
from math import *
import matplotlib.pyplot as plt
import os
# Where to save the figures and data files
PROJECT_ROOT_DIR = "Results"
FIGURE_ID = "Results/FigureFiles"
DATA_ID = "DataFiles/"
if not os.path.exists(PROJECT_ROOT_DIR):
os.mkdir(PROJECT_ROOT_DIR)
if not os.path.exists(FIGURE_ID):
os.makedirs(FIGURE_ID)
if not os.path.exists(DATA_ID):
os.makedirs(DATA_ID)
def image_path(fig_id):
return os.path.join(FIGURE_ID, fig_id)
def data_path(dat_id):
return os.path.join(DATA_ID, dat_id)
def save_fig(fig_id):
plt.savefig(image_path(fig_id) + ".png", format='png')
# Initial values, time step, positions and velocites
DeltaT = 0.0001
#set up arrays
tfinal = 100 # in years
n = ceil(tfinal/DeltaT)
# set up arrays for t, a, v, and x
t = np.zeros(n)
v = np.zeros((n,2))
r = np.zeros((n,2))
# setting up the kinetic, potential and total energy, note only functions of time
EKinetic = np.zeros(n)
EPotential = np.zeros(n)
ETotal = np.zeros(n)
# Initial conditions as compact 2-dimensional arrays
r0 = np.array([1.0,0.0])
v0 = np.array([0.0,2*pi])
r[0] = r0
v[0] = v0
Fourpi2 = 4*pi*pi
# Setting up variables for the calculation of energies
# distance that defines rabs in potential energy
rabs0 = sqrt(sum(r[0]*r[0]))
# Initial kinetic energy. Note that we skip the mass of the Earth here, that is MassEarth=1 in all codes
EKinetic[0] = 0.5*sum(v0*v0)
# Initial potential energy
EPotential[0] = -4*pi*pi/rabs0
# Initial total energy
ETotal[0] = EPotential[0]+EKinetic[0]
# Start integrating using Euler's method
for i in range(n-1):
# Set up the acceleration
# Here you could have defined your own function for this
rabs = sqrt(sum(r[i]*r[i]))
a = -Fourpi2*r[i]/(rabs**3)
# update velocity, time and position using Euler's forward method
v[i+1] = v[i] + DeltaT*a
# Only change when we add the Euler-Cromer method
r[i+1] = r[i] + DeltaT*v[i+1]
t[i+1] = t[i] + DeltaT
EKinetic[i+1] = 0.5*sum(v[i+1]*v[i+1])
EPotential[i+1] = -4*pi*pi/sqrt(sum(r[i+1]*r[i+1]))
ETotal[i+1] = EPotential[i+1]+EKinetic[i+1]
# Plot energies as functions of time
fig, axs = plt.subplots(3, 1)
axs[0].plot(t, EKinetic)
axs[0].set_xlim(0, tfinal)
axs[0].set_ylabel('Kinetic energy')
axs[1].plot(t, EPotential)
axs[1].set_ylabel('Potential Energy')
axs[2].plot(t, ETotal)
axs[2].set_xlabel('Time [yr]')
axs[2].set_ylabel('Total Energy')
fig.tight_layout()
save_fig("EarthSunEulerCromer")
plt.show()
Our final equations for the position and the velocity become then
$$ x_{i+1} = x_i+(\Delta t)v_i+\frac{(\Delta t)^2}{2}a_{i}+O((\Delta t)^3), $$and
$$ v_{i+1} = v_i+\frac{(\Delta t)}{2}\left(a_{i+1}+a_{i}\right)+O((\Delta t)^3). $$Note well that the term \( a_{i+1} \) depends on the position at \( x_{i+1} \). This means that you need to calculate the position at the updated time \( t_{i+1} \) before the computing the next velocity. Note also that the derivative of the velocity at the time \( t_i \) used in the updating of the position can be reused in the calculation of the velocity update as well.
We can now easily add the Verlet method to our original code as
DeltaT = 0.001
#set up arrays
tfinal = 100
n = ceil(tfinal/DeltaT)
# set up arrays for t, a, v, and x
t = np.zeros(n)
v = np.zeros((n,2))
r = np.zeros((n,2))
# Initial conditions as compact 2-dimensional arrays
r0 = np.array([1.0,0.0])
v0 = np.array([0.0,2*pi])
r[0] = r0
v[0] = v0
Fourpi2 = 4*pi*pi
# setting up the kinetic, potential and total energy, note only functions of time
EKinetic = np.zeros(n)
EPotential = np.zeros(n)
ETotal = np.zeros(n)
# Setting up variables for the calculation of energies
# distance that defines rabs in potential energy
rabs0 = sqrt(sum(r[0]*r[0]))
# Initial kinetic energy. Note that we skip the mass of the Earth here, that is MassEarth=1 in all codes
EKinetic[0] = 0.5*sum(v0*v0)
# Initial potential energy
EPotential[0] = -4*pi*pi/rabs0
# Initial total energy
ETotal[0] = EPotential[0]+EKinetic[0]
# Start integrating using the Velocity-Verlet method
for i in range(n-1):
# Set up forces, air resistance FD, note now that we need the norm of the vecto
# Here you could have defined your own function for this
rabs = sqrt(sum(r[i]*r[i]))
a = -Fourpi2*r[i]/(rabs**3)
# update velocity, time and position using the Velocity-Verlet method
r[i+1] = r[i] + DeltaT*v[i]+0.5*(DeltaT**2)*a
rabs = sqrt(sum(r[i+1]*r[i+1]))
anew = -4*(pi**2)*r[i+1]/(rabs**3)
v[i+1] = v[i] + 0.5*DeltaT*(a+anew)
t[i+1] = t[i] + DeltaT
EKinetic[i+1] = 0.5*sum(v[i+1]*v[i+1])
EPotential[i+1] = -4*pi*pi/sqrt(sum(r[i+1]*r[i+1]))
ETotal[i+1] = EPotential[i+1]+EKinetic[i+1]
# Plot energies as functions of time
fig, axs = plt.subplots(3, 1)
axs[0].plot(t, EKinetic)
axs[0].set_xlim(0, tfinal)
axs[0].set_ylabel('Kinetic energy')
axs[1].plot(t, EPotential)
axs[1].set_ylabel('Potential Energy')
axs[2].plot(t, ETotal)
axs[2].set_xlabel('Time [yr]')
axs[2].set_ylabel('Total Energy')
fig.tight_layout()
save_fig("EarthSunVelocityVerlet")
plt.show()
And we see that due to the smaller truncation error that energy conservation is improved as a function of time. Try out different time steps \( \Delta t \) and see if the results improve or worsen.


Front-end & Back-end
WebDev
Get Started
Fire-UI is an open source project built by developers, for developers. Fire UI is a user-friendly and reliable CSS Framework aimed to cope with any difficulties found by Web Developers on Front-End.
A Todo Application built using MERN Stack, which provides easier multi-tasking tasks.
ChatApp Realtime is a Web-Based Realtime Chat Application built using MERN Stack and Socket IO.
Fire-UI is an open source project built by developers, for developers. Fire UI is a user-friendly and reliable CSS Framework aimed to cope with any difficulties found by Web Developers on Front-End.
A Todo Application built using MERN Stack, which provides easier multi-tasking tasks.
ChatApp Realtime is a Web-Based Realtime Chat Application built using MERN Stack and Socket IO.
Multitude Developer, often abberviated as Multidev, a community where developers sharing and caring each other.
Any questions or just want to say hi? Feel free to fill the form or email me on stanleyowen06@gmail.com


| Active Certificates |
|
Active |
Product is certified and in good standing. |
|
Suspended by ONC |
Certification of the product has been suspended by ONC. While the product remains certified, the developer will be unable to update or certify new products during the suspension. |
|
Suspended by ONC-ACB |
Product's certification is suspended because corrective action plan not completed in time. The product is still considered certified, but it is at risk of having its certification withdrawn. |
| Decertified Products |
|
Terminated by ONC |
The certification of the product has been terminated by ONC. |
|
Withdrawn by Developer Under Surveillance/Review |
The certification of the product has been withdrawn by the developer while the product was under ONC-ACB surveillance or ONC direct review. It is no longer considered certified. |
|
Withdrawn by ONC-ACB |
Product's certification is withdrawn by the product's developer's ONC-ACB. No longer considered a certified product. |
| Inactive Certificates |
|
Withdrawn by Developer |
Product's certification is withdrawn by the product's developer. No longer a certified product. |
|
Retired |
Product's certification is retired as part of HHS policy. It is no longer certified. |


High Performance Computing Parameters Dialog¶
The HPC Parameters screen is specific to your institution. Figure 53 shows the parameters for Sandia’s cluster systems. Other than differences in the list of steps shown along the top, the same screen is used for all three data formats (Xyce, CSV, and HDF5).
WCID stands for Workload Characterization ID, which is required for job submission on the clusters. Because Slycat™ uses parallel processing to speed up Time Series model creation, users are required to have obtained a WCID prior to creating a Time Series model. Your WCID is associated with an Strategic Management Unit/Program Management Unit (SMU/PMU), which is used to specify the Partition or queue-name. At Sandia, the choices are nw, ec, dsa, ihns, ldrd, cee, viz, or viz batch).


This demo introduces Tag Helpers and how they can be used as an alternative to HTML Helpers. Additionally, it shows how to create custom Tag Helpers.
Tag Helpers are a new feature introduced in ASP.NET Core 1.0. They combine the power of the Razor view engine and the expressiveness of HTML by allowing you to write HTML tags in your views rather than inline C# - all this with IntelliSense support.
In this demo, you will see how to:
Follow these steps to set up your environment for the demo.
This demo is composed of the following segments:
In Visual Studio, go to File | New | Project.
In the Templates | Visual C# | Web tab, select the ASP.NET Web Application project. Name it TagHelpersDemo.
Creating a web project
From the ASP.NET 5 Templates list, select the Web Application template and click OK.
Selecting the Web Site template
Open Views/Account/Register.cshtml.
Look at the Tag Helpers being used in this view (purple and bold) and play around with setting their attributes and exploring the IntelliSense offered for the different attribute types.
Showing the Register view
Run the application and go to the Register page. Using the Microsoft Edge development tools, look at the HTML output of the Tag Helpers.
Showing the Register View's HTML output
Switch back to Visual Studio. You can take a look at the other views in the Views/Account folder to see how they use Tag Helpers.
Now, open the Views/Shared/_Layout.cshtml file.
Look at the Tag Helpers being used in the element to render CSS stylesheets and at the end of the page to render JavaScript files.
Showing Tag Helpers in the Layout
In Microsoft Edge, look at the HTML output of the CSS files.
Showing the HTML output of the Layout
Create a new RepeatTagHelper class in the root of the application you created in the previous segment by adding a new item to the project and selecting Razor Tag Helper.
Creating the Tag Helper class
Show the content of the generated file and update the HtmlTargetElement
parameter to repeat
.
Showing the content of the generated file
Replace the content of the RepeatTagHelper class with the following code snippet.
public int Count { get; set; }
public override async Task ProcessAsync(TagHelperContext context, TagHelperOutput output)
{
}
Add the following code to the body of the ProcessAsync method that repeats the content via the output parameter in a loop of Count iterations.
for (int i = 0; i < Count; i++)
{
output.Content.Append(await output.GetChildContentAsync());
}
Open the Views/_ViewImports.cshtml file and add a line at the end to register the assembly that contains your Tag Helper.
@addTagHelper "*, TagHelpersDemo"
Open the Views/Home/Index.cshtml file and use your Tag Helper by adding the following code directly above the

HTML element.
<repeat count="5">

I'll be repeated!!

</repeat>
Using the custom tag helper
Run the application again and ensure your Tag Helper is working.
Note that the Tag Helper is rendering itself as a
Showing the custom tag helper
Open the RepeatTagHelper class, and at the end of the ProcessAsync method add a line to null the tag name.
output.TagName = null;
Run the application again and notice that the outer tag is no longer being rendered.
The custom tag is no longer rendered
In this demo you have walked through the use and creation of ASP.NET Core Tag Helpers and seen how IntelliSense provides support for them.


Vinyl Destination
Home
My Collection
Search
Log Out
Today's top Pop albums on last.fm:
Login
Login
Not registered yet? Sign up
here
!