This guide provides an overview of the HDFS High Availability (HA) feature and how to configure and manage an HA HDFS cluster, using NFS for the shared storage required by the NameNodes.
This document assumes that the reader has a general understanding of general components and node types in an HDFS cluster. Please refer to the HDFS Architecture guide for details.
This guide discusses how to configure and use HDFS HA using a shared NFS directory to share edit logs between the Active and Standby NameNodes. For information on how to configure HDFS HA using the Quorum Journal Manager instead of NFS, please see this alternative guide.
Prior to Hadoop 2.0.0, the NameNode was a single point of failure (SPOF) in an HDFS cluster. Each cluster had a single NameNode, and if that machine or process became unavailable, the cluster as a whole would be unavailable until the NameNode was either restarted or brought up on a separate machine.
This impacted the total availability of the HDFS cluster in two major ways:
In the case of an unplanned event such as a machine crash, the cluster would be unavailable until an operator restarted the NameNode.
Planned maintenance events such as software or hardware upgrades on the NameNode machine would result in windows of cluster downtime.
The HDFS High Availability feature addresses the above problems by providing the option of running two redundant NameNodes in the same cluster in an Active/Passive configuration with a hot standby. This allows a fast failover to a new NameNode in the case that a machine crashes, or a graceful administrator-initiated failover for the purpose of planned maintenance.
In a typical HA cluster, two separate machines are configured as NameNodes. At any point in time, exactly one of the NameNodes is in an Active state, and the other is in a Standby state. The Active NameNode is responsible for all client operations in the cluster, while the Standby is simply acting as a slave, maintaining enough state to provide a fast failover if necessary.
In order for the Standby node to keep its state synchronized with the Active node, the current implementation requires that the two nodes both have access to a directory on a shared storage device (eg an NFS mount from a NAS). This restriction will likely be relaxed in future versions.
When any namespace modification is performed by the Active node, it durably logs a record of the modification to an edit log file stored in the shared directory. The Standby node is constantly watching this directory for edits, and as it sees the edits, it applies them to its own namespace. In the event of a failover, the Standby will ensure that it has read all of the edits from the shared storage before promoting itself to the Active state. This ensures that the namespace state is fully synchronized before a failover occurs.
In order to provide a fast failover, it is also necessary that the Standby node have up-to-date information regarding the location of blocks in the cluster. In order to achieve this, the DataNodes are configured with the location of both NameNodes, and send block location information and heartbeats to both.
It is vital for the correct operation of an HA cluster that only one of the NameNodes be Active at a time. Otherwise, the namespace state would quickly diverge between the two, risking data loss or other incorrect results. In order to ensure this property and prevent the so-called “split-brain scenario,” the administrator must configure at least one fencing method for the shared storage. During a failover, if it cannot be verified that the previous Active node has relinquished its Active state, the fencing process is responsible for cutting off the previous Active’s access to the shared edits storage. This prevents it from making any further edits to the namespace, allowing the new Active to safely proceed with failover.
In order to deploy an HA cluster, you should prepare the following:
NameNode machines - the machines on which you run the Active and Standby NameNodes should have equivalent hardware to each other, and equivalent hardware to what would be used in a non-HA cluster.
Shared storage - you will need to have a shared directory which both NameNode machines can have read/write access to. Typically this is a remote filer which supports NFS and is mounted on each of the NameNode machines. Currently only a single shared edits directory is supported. Thus, the availability of the system is limited by the availability of this shared edits directory, and therefore in order to remove all single points of failure there needs to be redundancy for the shared edits directory. Specifically, multiple network paths to the storage, and redundancy in the storage itself (disk, network, and power). Beacuse of this, it is recommended that the shared storage server be a high-quality dedicated NAS appliance rather than a simple Linux server.
Note that, in an HA cluster, the Standby NameNode also performs checkpoints of the namespace state, and thus it is not necessary to run a Secondary NameNode, CheckpointNode, or BackupNode in an HA cluster. In fact, to do so would be an error. This also allows one who is reconfiguring a non-HA-enabled HDFS cluster to be HA-enabled to reuse the hardware which they had previously dedicated to the Secondary NameNode.
Similar to Federation configuration, HA configuration is backward compatible and allows existing single NameNode configurations to work without change. The new configuration is designed such that all the nodes in the cluster may have the same configuration without the need for deploying different configuration files to different machines based on the type of the node.
Like HDFS Federation, HA clusters reuse the nameservice ID to identify a single HDFS instance that may in fact consist of multiple HA NameNodes. In addition, a new abstraction called NameNode ID is added with HA. Each distinct NameNode in the cluster has a different NameNode ID to distinguish it. To support a single configuration file for all of the NameNodes, the relevant configuration parameters are suffixed with the nameservice ID as well as the NameNode ID.
To configure HA NameNodes, you must add several configuration options to your hdfs-site.xml configuration file.
The order in which you set these configurations is unimportant, but the values you choose for dfs.nameservices and dfs.ha.namenodes.[nameservice ID] will determine the keys of those that follow. Thus, you should decide on these values before setting the rest of the configuration options.
dfs.nameservices - the logical name for this new nameservice
Choose a logical name for this nameservice, for example “mycluster”, and use this logical name for the value of this config option. The name you choose is arbitrary. It will be used both for configuration and as the authority component of absolute HDFS paths in the cluster.
Note: If you are also using HDFS Federation, this configuration setting should also include the list of other nameservices, HA or otherwise, as a comma-separated list.
<property> <name>dfs.nameservices</name> <value>mycluster</value> </property>
dfs.ha.namenodes.[nameservice ID] - unique identifiers for each NameNode in the nameservice
Configure with a list of comma-separated NameNode IDs. This will be used by DataNodes to determine all the NameNodes in the cluster. For example, if you used “mycluster” as the nameservice ID previously, and you wanted to use “nn1” and “nn2” as the individual IDs of the NameNodes, you would configure this as such:
<property> <name>dfs.ha.namenodes.mycluster</name> <value>nn1,nn2</value> </property>
Note: Currently, only a maximum of two NameNodes may be configured per nameservice.
dfs.namenode.rpc-address.[nameservice ID].[name node ID] - the fully-qualified RPC address for each NameNode to listen on
For both of the previously-configured NameNode IDs, set the full address and IPC port of the NameNode processs. Note that this results in two separate configuration options. For example:
<property> <name>dfs.namenode.rpc-address.mycluster.nn1</name> <value>machine1.example.com:8020</value> </property> <property> <name>dfs.namenode.rpc-address.mycluster.nn2</name> <value>machine2.example.com:8020</value> </property>
Note: You may similarly configure the “servicerpc-address” setting if you so desire.
dfs.namenode.http-address.[nameservice ID].[name node ID] - the fully-qualified HTTP address for each NameNode to listen on
Similarly to rpc-address above, set the addresses for both NameNodes’ HTTP servers to listen on. For example:
<property> <name>dfs.namenode.http-address.mycluster.nn1</name> <value>machine1.example.com:50070</value> </property> <property> <name>dfs.namenode.http-address.mycluster.nn2</name> <value>machine2.example.com:50070</value> </property>
Note: If you have Hadoop’s security features enabled, you should also set the https-address similarly for each NameNode.
dfs.namenode.shared.edits.dir - the location of the shared storage directory
This is where one configures the path to the remote shared edits directory which the Standby NameNode uses to stay up-to-date with all the file system changes the Active NameNode makes. You should only configure one of these directories. This directory should be mounted r/w on both NameNode machines. The value of this setting should be the absolute path to this directory on the NameNode machines. For example:
<property> <name>dfs.namenode.shared.edits.dir</name> <value>file:///mnt/filer1/dfs/ha-name-dir-shared</value> </property>
dfs.client.failover.proxy.provider.[nameservice ID] - the Java class that HDFS clients use to contact the Active NameNode
Configure the name of the Java class which will be used by the DFS Client to determine which NameNode is the current Active, and therefore which NameNode is currently serving client requests. The only implementation which currently ships with Hadoop is the ConfiguredFailoverProxyProvider, so use this unless you are using a custom one. For example:
<property> <name>dfs.client.failover.proxy.provider.mycluster</name> <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value> </property>
dfs.ha.fencing.methods - a list of scripts or Java classes which will be used to fence the Active NameNode during a failover
It is critical for correctness of the system that only one NameNode be in the Active state at any given time. Thus, during a failover, we first ensure that the Active NameNode is either in the Standby state, or the process has terminated, before transitioning the other NameNode to the Active state. In order to do this, you must configure at least one fencing method. These are configured as a carriage-return-separated list, which will be attempted in order until one indicates that fencing has succeeded. There are two methods which ship with Hadoop: shell and sshfence. For information on implementing your own custom fencing method, see the org.apache.hadoop.ha.NodeFencer class.
sshfence - SSH to the Active NameNode and kill the process
The sshfence option SSHes to the target node and uses fuser to kill the process listening on the service’s TCP port. In order for this fencing option to work, it must be able to SSH to the target node without providing a passphrase. Thus, one must also configure the dfs.ha.fencing.ssh.private-key-files option, which is a comma-separated list of SSH private key files. For example:
<property> <name>dfs.ha.fencing.methods</name> <value>sshfence</value> </property> <property> <name>dfs.ha.fencing.ssh.private-key-files</name> <value>/home/exampleuser/.ssh/id_rsa</value> </property>
Optionally, one may configure a non-standard username or port to perform the SSH. One may also configure a timeout, in milliseconds, for the SSH, after which this fencing method will be considered to have failed. It may be configured like so:
<property> <name>dfs.ha.fencing.methods</name> <value>sshfence([[username][:port]])</value> </property> <property> <name>dfs.ha.fencing.ssh.connect-timeout</name> <value>30000</value> </property>
shell - run an arbitrary shell command to fence the Active NameNode
The shell fencing method runs an arbitrary shell command. It may be configured like so:
<property> <name>dfs.ha.fencing.methods</name> <value>shell(/path/to/my/script.sh arg1 arg2 ...)</value> </property>
The string between ‘(’ and ‘)’ is passed directly to a bash shell and may not include any closing parentheses.
The shell command will be run with an environment set up to contain all of the current Hadoop configuration variables, with the ‘_’ character replacing any ‘.’ characters in the configuration keys. The configuration used has already had any namenode-specific configurations promoted to their generic forms – for example dfs_namenode_rpc-address will contain the RPC address of the target node, even though the configuration may specify that variable as dfs.namenode.rpc-address.ns1.nn1.
Additionally, the following variables referring to the target node to be fenced are also available:
| $target_host | hostname of the node to be fenced |
| $target_port | IPC port of the node to be fenced |
| $target_address | the above two, combined as host:port |
| $target_nameserviceid | the nameservice ID of the NN to be fenced |
| $target_namenodeid | the namenode ID of the NN to be fenced |
These environment variables may also be used as substitutions in the shell command itself. For example:
<property> <name>dfs.ha.fencing.methods</name> <value>shell(/path/to/my/script.sh --nameservice=$target_nameserviceid $target_host:$target_port)</value> </property>
If the shell command returns an exit code of 0, the fencing is determined to be successful. If it returns any other exit code, the fencing was not successful and the next fencing method in the list will be attempted.
Note: This fencing method does not implement any timeout. If timeouts are necessary, they should be implemented in the shell script itself (eg by forking a subshell to kill its parent in some number of seconds).
fs.defaultFS - the default path prefix used by the Hadoop FS client when none is given
Optionally, you may now configure the default path for Hadoop clients to use the new HA-enabled logical URI. If you used “mycluster” as the nameservice ID earlier, this will be the value of the authority portion of all of your HDFS paths. This may be configured like so, in your core-site.xml file:
<property> <name>fs.defaultFS</name> <value>hdfs://mycluster</value> </property>
After all of the necessary configuration options have been set, one must initially synchronize the two HA NameNodes’ on-disk metadata.
If you are setting up a fresh HDFS cluster, you should first run the format command (hdfs namenode -format) on one of NameNodes.
If you have already formatted the NameNode, or are converting a non-HA-enabled cluster to be HA-enabled, you should now copy over the contents of your NameNode metadata directories to the other, unformatted NameNode by running the command “hdfs namenode -bootstrapStandby” on the unformatted NameNode. Running this command will also ensure that the shared edits directory (as configured by dfs.namenode.shared.edits.dir) contains sufficient edits transactions to be able to start both NameNodes.
If you are converting a non-HA NameNode to be HA, you should run the command “hdfs -initializeSharedEdits”, which will initialize the shared edits directory with the edits data from the local NameNode edits directories.
At this point you may start both of your HA NameNodes as you normally would start a NameNode.
You can visit each of the NameNodes’ web pages separately by browsing to their configured HTTP addresses. You should notice that next to the configured address will be the HA state of the NameNode (either “standby” or “active”.) Whenever an HA NameNode starts, it is initially in the Standby state.
Now that your HA NameNodes are configured and started, you will have access to some additional commands to administer your HA HDFS cluster. Specifically, you should familiarize yourself with all of the subcommands of the “hdfs haadmin” command. Running this command without any additional arguments will display the following usage information:
Usage: DFSHAAdmin [-ns <nameserviceId>] [-transitionToActive <serviceId>] [-transitionToStandby <serviceId>] [-failover [--forcefence] [--forceactive] <serviceId> <serviceId>] [-getServiceState <serviceId>] [-checkHealth <serviceId>] [-help <command>]
This guide describes high-level uses of each of these subcommands. For specific usage information of each subcommand, you should run “hdfs haadmin -help <command>”.
transitionToActive and transitionToStandby - transition the state of the given NameNode to Active or Standby
These subcommands cause a given NameNode to transition to the Active or Standby state, respectively. These commands do not attempt to perform any fencing, and thus should rarely be used. Instead, one should almost always prefer to use the “hdfs haadmin -failover” subcommand.
failover - initiate a failover between two NameNodes
This subcommand causes a failover from the first provided NameNode to the second. If the first NameNode is in the Standby state, this command simply transitions the second to the Active state without error. If the first NameNode is in the Active state, an attempt will be made to gracefully transition it to the Standby state. If this fails, the fencing methods (as configured by dfs.ha.fencing.methods) will be attempted in order until one succeeds. Only after this process will the second NameNode be transitioned to the Active state. If no fencing method succeeds, the second NameNode will not be transitioned to the Active state, and an error will be returned.
getServiceState - determine whether the given NameNode is Active or Standby
Connect to the provided NameNode to determine its current state, printing either “standby” or “active” to STDOUT appropriately. This subcommand might be used by cron jobs or monitoring scripts which need to behave differently based on whether the NameNode is currently Active or Standby.
checkHealth - check the health of the given NameNode
Connect to the provided NameNode to check its health. The NameNode is capable of performing some diagnostics on itself, including checking if internal services are running as expected. This command will return 0 if the NameNode is healthy, non-zero otherwise. One might use this command for monitoring purposes.
Note: This is not yet implemented, and at present will always return success, unless the given NameNode is completely down.
The above sections describe how to configure manual failover. In that mode, the system will not automatically trigger a failover from the active to the standby NameNode, even if the active node has failed. This section describes how to configure and deploy automatic failover.
Automatic failover adds two new components to an HDFS deployment: a ZooKeeper quorum, and the ZKFailoverController process (abbreviated as ZKFC).
Apache ZooKeeper is a highly available service for maintaining small amounts of coordination data, notifying clients of changes in that data, and monitoring clients for failures. The implementation of automatic HDFS failover relies on ZooKeeper for the following things:
Failure detection - each of the NameNode machines in the cluster maintains a persistent session in ZooKeeper. If the machine crashes, the ZooKeeper session will expire, notifying the other NameNode that a failover should be triggered.
Active NameNode election - ZooKeeper provides a simple mechanism to exclusively elect a node as active. If the current active NameNode crashes, another node may take a special exclusive lock in ZooKeeper indicating that it should become the next active.
The ZKFailoverController (ZKFC) is a new component which is a ZooKeeper client which also monitors and manages the state of the NameNode. Each of the machines which runs a NameNode also runs a ZKFC, and that ZKFC is responsible for:
Health monitoring - the ZKFC pings its local NameNode on a periodic basis with a health-check command. So long as the NameNode responds in a timely fashion with a healthy status, the ZKFC considers the node healthy. If the node has crashed, frozen, or otherwise entered an unhealthy state, the health monitor will mark it as unhealthy.
ZooKeeper session management - when the local NameNode is healthy, the ZKFC holds a session open in ZooKeeper. If the local NameNode is active, it also holds a special “lock” znode. This lock uses ZooKeeper’s support for “ephemeral” nodes; if the session expires, the lock node will be automatically deleted.
ZooKeeper-based election - if the local NameNode is healthy, and the ZKFC sees that no other node currently holds the lock znode, it will itself try to acquire the lock. If it succeeds, then it has “won the election”, and is responsible for running a failover to make its local NameNode active. The failover process is similar to the manual failover described above: first, the previous active is fenced if necessary, and then the local NameNode transitions to active state.
For more details on the design of automatic failover, refer to the design document attached to HDFS-2185 on the Apache HDFS JIRA.
In a typical deployment, ZooKeeper daemons are configured to run on three or five nodes. Since ZooKeeper itself has light resource requirements, it is acceptable to collocate the ZooKeeper nodes on the same hardware as the HDFS NameNode and Standby Node. Many operators choose to deploy the third ZooKeeper process on the same node as the YARN ResourceManager. It is advisable to configure the ZooKeeper nodes to store their data on separate disk drives from the HDFS metadata for best performance and isolation.
The setup of ZooKeeper is out of scope for this document. We will assume that you have set up a ZooKeeper cluster running on three or more nodes, and have verified its correct operation by connecting using the ZK CLI.
 It is not currently possible to transition from a manual failover setup to an automatic failover setup while the cluster is running.
The configuration of automatic failover requires the addition of two new parameters to your configuration. In your hdfs-site.xml file, add:
<property> <name>dfs.ha.automatic-failover.enabled</name> <value>true</value> </property>
This specifies that the cluster should be set up for automatic failover. In your core-site.xml file, add:
<property> <name>ha.zookeeper.quorum</name> <value>zk1.example.com:2181,zk2.example.com:2181,zk3.example.com:2181</value> </property>
This lists the host-port pairs running the ZooKeeper service.
As with the parameters described earlier in the document, these settings may be configured on a per-nameservice basis by suffixing the configuration key with the nameservice ID. For example, in a cluster with federation enabled, you can explicitly enable automatic failover for only one of the nameservices by setting dfs.ha.automatic-failover.enabled.my-nameservice-id.
There are also several other configuration parameters which may be set to control the behavior of automatic failover; however, they are not necessary for most installations. Please refer to the configuration key specific documentation for details.
After the configuration keys have been added, the next step is to initialize required state in ZooKeeper. You can do so by running the following command from one of the NameNode hosts.
[hdfs]$ $HADOOP_PREFIX/bin/zkfc -formatZK
This will create a znode in ZooKeeper inside of which the automatic failover system stores its data.
Since automatic failover has been enabled in the configuration, the start-dfs.sh script will now automatically start a ZKFC daemon on any machine that runs a NameNode. When the ZKFCs start, they will automatically select one of the NameNodes to become active.
If you manually manage the services on your cluster, you will need to manually start the zkfc daemon on each of the machines that runs a NameNode. You can start the daemon by running:
[hdfs]$ $HADOOP_PREFIX/sbin/hadoop-daemon.sh --script $HADOOP_PREFIX/bin/hdfs start zkfc
If you are running a secure cluster, you will likely want to ensure that the information stored in ZooKeeper is also secured. This prevents malicious clients from modifying the metadata in ZooKeeper or potentially triggering a false failover.
In order to secure the information in ZooKeeper, first add the following to your core-site.xml file:
<property> <name>ha.zookeeper.auth</name><EMAIL_ADDRESS></property> <property> <name>ha.zookeeper.acl</name><EMAIL_ADDRESS></property>
Please note the ‘@’ character in these values – this specifies that the configurations are not inline, but rather point to a file on disk.
The first configured file specifies a list of ZooKeeper authentications, in the same format as used by the ZK CLI. For example, you may specify something like:
digest:hdfs-zkfcs:mypassword
…where hdfs-zkfcs is a unique username for ZooKeeper, and mypassword is some unique string used as a password.
Next, generate a ZooKeeper ACL that corresponds to this authentication, using a command like the following:
[hdfs]$ java -cp $ZK_HOME/lib/*:$ZK_HOME/zookeeper-3.4.2.jar org.apache.zookeeper.server.auth.DigestAuthenticationProvider hdfs-zkfcs:mypassword output: hdfs-zkfcs:mypassword->hdfs-zkfcs:P/OQvnYyU/nF/mGYvB/xurX8dYs=
Copy and paste the section of this output after the ‘->’ string into the file zk-acls.txt, prefixed by the string “digest:”. For example:
digest:hdfs-zkfcs:vlUvLnd8MlacsE80rDuu6ONESbM=:rwcda
In order for these ACLs to take effect, you should then rerun the zkfc -formatZK command as described above.
After doing so, you may verify the ACLs from the ZK CLI as follows:
[zk: localhost:2181(CONNECTED) 1] getAcl /hadoop-ha 'digest,'hdfs-zkfcs:vlUvLnd8MlacsE80rDuu6ONESbM= : cdrwa
Once automatic failover has been set up, you should test its operation. To do so, first locate the active NameNode. You can tell which node is active by visiting the NameNode web interfaces – each node reports its HA state at the top of the page.
Once you have located your active NameNode, you may cause a failure on that node. For example, you can use kill -9 <pid of NN> to simulate a JVM crash. Or, you could power cycle the machine or unplug its network interface to simulate a different kind of outage. After triggering the outage you wish to test, the other NameNode should automatically become active within several seconds. The amount of time required to detect a failure and trigger a fail-over depends on the configuration of ha.zookeeper.session-timeout.ms, but defaults to 5 seconds.
If the test does not succeed, you may have a misconfiguration. Check the logs for the zkfc daemons as well as the NameNode daemons in order to further diagnose the issue.
Is it important that I start the ZKFC and NameNode daemons in any particular order?
No. On any given node you may start the ZKFC before or after its corresponding NameNode.
What additional monitoring should I put in place?
You should add monitoring on each host that runs a NameNode to ensure that the ZKFC remains running. In some types of ZooKeeper failures, for example, the ZKFC may unexpectedly exit, and should be restarted to ensure that the system is ready for automatic failover.
Additionally, you should monitor each of the servers in the ZooKeeper quorum. If ZooKeeper crashes, then automatic failover will not function.
What happens if ZooKeeper goes down?
If the ZooKeeper cluster crashes, no automatic failovers will be triggered. However, HDFS will continue to run without any impact. When ZooKeeper is restarted, HDFS will reconnect with no issues.
Can I designate one of my NameNodes as primary/preferred?
No. Currently, this is not supported. Whichever NameNode is started first will become active. You may choose to start the cluster in a specific order such that your preferred node starts first.
How can I initiate a manual failover when automatic failover is configured?
Even if automatic failover is configured, you may initiate a manual failover using the same hdfs haadmin command. It will perform a coordinated failover.
One option for shared storage for the NameNode is BookKeeper. BookKeeper achieves high availability and strong durability guarantees by replicating edit log entries across multiple storage nodes. The edit log can be striped across the storage nodes for high performance. Fencing is supported in the protocol, i.e, BookKeeper will not allow two writers to write the single edit log.
The meta data for BookKeeper is stored in ZooKeeper. In current HA architecture, a Zookeeper cluster is required for ZKFC. The same cluster can be for BookKeeper metadata.
For more details on building a BookKeeper cluster, please refer to the BookKeeper documentation
The BookKeeperJournalManager is an implementation of the HDFS JournalManager interface, which allows custom write ahead logging implementations to be plugged into the HDFS NameNode.
BookKeeper Journal Manager
To use BookKeeperJournalManager, add the following to hdfs-site.xml.
<property> <name>dfs.namenode.shared.edits.dir</name> <value>bookkeeper://zk1:2181;zk2:2181;zk3:2181/hdfsjournal</value> </property> <property> <name>dfs.namenode.edits.journal-plugin.bookkeeper</name> <value>org.apache.hadoop.contrib.bkjournal.BookKeeperJournalManager</value> </property>
The URI format for bookkeeper is bookkeeper://[zkEnsemble]/[rootZnode] [zookkeeper ensemble] is a list of semi-colon separated, zookeeper host:port pairs. In the example above there are 3 servers, in the ensemble, zk1, zk2 & zk3, each one listening on port 2181.
[root znode] is the path of the zookeeper znode, under which the edit log information will be stored.
The class specified for the journal-plugin must be available in the NameNode’s classpath. We explain how to generate a jar file with the journal manager and its dependencies, and how to put it into the classpath below.
More configuration options
<property> <name>dfs.namenode.bookkeeperjournal.output-buffer-size</name> <value>1024</value> </property>
<property> <name>dfs.namenode.bookkeeperjournal.ensemble-size</name> <value>3</value> </property>
<property> <name>dfs.namenode.bookkeeperjournal.quorum-size</name> <value>2</value> </property>
<property> <name>dfs.namenode.bookkeeperjournal.digestPw</name> <value>myPassword</value> </property>
<property> <name>dfs.namenode.bookkeeperjournal.zk.session.timeout</name> <value>3000</value> </property>
Building BookKeeper Journal Manager plugin jar
To generate the distribution packages for BK journal, do the following.
$ mvn clean package -Pdist
This will generate a jar with the BookKeeperJournalManager, hadoop-hdfs/src/contrib/bkjournal/target/hadoop-hdfs-bkjournal-VERSION.jar
Note that the -Pdist part of the build command is important, this would copy the dependent bookkeeper-server jar under hadoop-hdfs/src/contrib/bkjournal/target/lib.
Putting the BookKeeperJournalManager in the NameNode classpath
To run a HDFS namenode using BookKeeper as a backend, copy the bkjournal and bookkeeper-server jar, mentioned above, into the lib directory of hdfs. In the standard distribution of HDFS, this is at $HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/
cp hadoop-hdfs/src/contrib/bkjournal/target/hadoop-hdfs-bkjournal-VERSION.jar $HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/
Current limitations
1) Security in BookKeeper. BookKeeper does not support SASL nor SSL for connections between the NameNode and BookKeeper storage nodes.


main Specifies the main content of a document
a Defines a hyperlinkabbr Defines an abbreviation
address Defines contact information for the author / owner of a document
article Defines an articleb Defines bold textbdi Isolates a part of text that might be formatted in a different direction from other text outside it
bdo Overrides the current text direction
blockquote Defines a section that is quoted from another source
cite Defines the title of a work
caption Defines a table caption
code Defines a piece of computer code
col Specifies column properties for each column within a colgroup element
colgroup Specifies a group of one or more columns in a table for formatting
del Defines text that has been deleted from a documentdetails Defines additional details that the user can view or hidedfn Defines a definition term
dt Defines a term / name in a description list
dd Defines a description / value of a term in a description list
dt Defines a term / name in a description list
dd Defines a description / value of a term in a description list
em Defines emphasized texthr Defines a thematic change in the content
i Defines a part of text in an alternate voice or moodins Defines a text that has been inserted into a document
kbd Defines keyboard inputlink Defines the relationship between a document and an external resource (most used to link to style sheets)
mark Defines marked / highlighted text
meter Defines a scalar measurement within a known range (a gauge)
p Defines a paragraph
param Defines a parameter for an object
pre Defines pre-formatted text
q Defines a short quotationruby Defines a ruby annotation (for East Asian typography)
s Defines text that is no longer correctsamp Defines sample output from a computer programsection Defines a section in a documentsmall Defines smaller text
span Defines a section in a documentstrong Defines important textsub Defines subscripted textsummary Defines a visible heading for a details elementsup Defines superscripted text
Heading
table Defines a table
Footing
u Defines text that should be stylistically different from normal textvar Defines a variable2 + 2 = Wbr Defines a possible line-break


Home
About
Contact
Just Ask
Pregnancy Helpline
SMS Data Repository
Sort
Phone Number
End Result
Risk
880-10-1000-1000
Heartburn
Low
880-11-1111-1111
Shortness of Breath
Medium
880-12-2222-2222
Faint
High
880-12-2222-2222
Severe bleeding
High
Phone Number
# of High Risk Queries
880-12-2222-2222
2


HTML5 Document Structure Example
This page should be tried in safari, chrome or Mozila.
Home
News
Contact
About
Login
Once article can have multiple sections


Decentralized Library
Your balance:
Recipient:
Amount:
MyBalance
Give
Scrappy
Author
:
Golden Retriever
Price
:
3
Location
:
Warren, MI
#Left
:
0
Borrow
Return


Bypolls to three Bengal Assembly seats today
This is the first time that the Left Front and the Congress have tied up officially after the 2016 Assembly poll.
by Special CorrespondentBypolls to three Assembly seats — Kharagpur Sadar and Karimpur, in south Bengal, and Kaliaganj in north Bengal — will be held on Monday. The results will be declared on November 28.
The elections are significant for three reasons. This is the first time that the Left Front and the Congress have tied up officially after the 2016 Assembly poll.
Also, they are the first elections following the Ayodhya verdict and the Union Home Minister’s announcement of a countrywide National Register of Citizens.
The Congress has put up Left-backed candidates in Kaliaganj and Kharagpur Sadar. The Kaliaganj Assembly segment, once considered a Congress bastion led by former Union Minister Priya Ranjan Dasmunshi, gave the BJP a huge lead in the 2019 Lok Sabha poll. The Congress, backed by the Left, had won the seat in 2016.
In 2016, BJP State president Dilip Ghosh had won the Kharagpur Sadar seat by a slender margin. The Trinamool Congress finished a distant third. However, Mr. Ghosh polled 58% of the votes in the [Kharagpur Sadar] segment in 2019. The Trinamool won Karimpur with about 15,000 votes over its nearst rival in both the 2016 and 2019 polls. The Left Front has put up its candidate, backed by the Congress, in the seat now.


Server Error
Sorry, but Flip.TV is experiencing very heavy traffic.
It appears we too busy to serve your request. Please try back in a few minutes.


SurveyMonkeyComposition
public struct SurveyMonkeyComposition
Styling of SurveyMonkeyComponent
.
Example view of component:
-
Title displayed on the top of component.
Default value: System font of size 18, color: #0C1E3DDeclaration
Swift
public var titleFont: FontComposition
-
Footer displayed on the bottom of component.
Default value: System font of size 12, color: #0C1E3DDeclaration
Swift
public var footerFont: FontComposition
-
Submit button style.
Note: Text displayed on the button is fetched from wingu portal
Default is defaultWinguButtonComposition
style.Declaration
Swift
public var actionButton: WinguButtonComposition
-
Background color for whole component.
Default value #0C1E3D, alpha: 0.1Declaration
Swift
public var backgroundColor: UIColor
-
Localized strings of
SurveyMonkeyComponent
.Declaration
Swift
public var translations: SurveyMonkeyTranslation
-
The default composition for
SurveyMonkeyComposition
Check the variables declaration to receive the default values for every parameter.Declaration
Swift
public static let defaultComposition: SurveyMonkeyComposition


--- layout: archive title: Zulip Chat Archive permalink: /stream/113488-general/topic/core.html ---
What is the core/ directory in mathlib for? Is it stuff that goes "under" tactics?
core/
Last updated: Apr 24 2020 at 19:02 UTC


com.pzj.core.smp.write
接口
BlackListWriteMapper
FlowControlWriteMapper
SmsChannelUserinfoWriteMapper
SmsChannelWriteMapper
SmsCountWriteMapper
SmsRecordWriteMapper


The iframe and Ajax approach will both use the URL in the distribution window.
The JavaScript approach involves copying and pasting the code.
The iframe and Ajax approach will both use the URL in the distribution window.
The JavaScript approach involves copying and pasting the code.


The purpose of this website is to provide regular, timely and relevant information to the people of British Columbia about the efforts being undertaken by the Commission.
The hearings have concluded.
Please click on the links below for:
Since the Commission was announced in May 2019, it has held five community meetings around the province, undertaken thousands of hours of investigation, research and interviews, held 143 hearing sessions over 138 hearing days, heard testimony from 199 witnesses with an additional 23 witnesses providing evidence via sworn affidavit, received 1,063 exhibits and released its Interim Report.


Toggle navigation
Menu
LANY
Tour
Merch
Lyrics
This Is LANY
TOUR
DATES
yeah babe, no way
kinda
4EVER!
I Loved You.
ILYSB
Make Out
pink skies
kinda
Someone Else
I Loved You.
Bad, Bad, Bad
Make Out
Keep Up!
Get the
Newsletter.


Check out the new feature on aerodynamics in Microsoft Flight Simulator
A new discovery feature takes flight in examining aerodynamics.
by Samuel TolbertSource: Xbox Game Studios (screenshot)
What you need to know
- Microsoft Flight Simulator is currently being developed by Asobo Studio and Xbox Game Studios Publishing.
- A new video feature goes over the aerodynamics in the game.
- Microsoft Flight Simulator is currently set to release sometime in 2020.
- It'll be included in Xbox Game Pass at launch.
First revealed at E3 2019, Microsoft Flight Simulator aims to recapture the audience that loved the long-running series of flight simulation titles. After giving a small roadmap earlier this year, the team at working on the game has continued to give a series of updates and videos on different elements of the game's design. The most recent of these is a new video in which Sebastian Wloch, co-founder and CEO of Asobo Studio, talkes about aerodynamics in Microsoft Flight Simulator.
If you're interested in testing this game ahead of launch, be sure to sign up for the insider program. Microsoft Flight Simulator is being developed by Asobo Studio, in partnership with a team at Xbox Game Studios Publishing. Currently, the game is set to release sometime in 2020. Like all Xbox Game Studios titles, it'll be included in Xbox Game Pass at launch.
Related: Microsoft Flight Simulator lead on PC cloud power, Xbox One debut, mods and updates
Best of the best
Xbox Game Pass Ultimate
Xbox Game Pass and Xbox Live Gold in one
Xbox Game Pass gives you access to over a hundred games for one monthly fee. Xbox Game Pass Ultimate also adds Xbox Live Gold to the package so you can play online with your friends. Right now, you can purchase a three-month subscription for $45 through Amazon.
Get more Xbox
Xbox
- Xbox One S vs. Xbox One X: Which should you buy?
- Best Xbox One X and Xbox One S Accessories
- Best Xbox One Headsets 2019
- Best Xbox One Wireless Headsets 2019
- Best Xbox Controller Charging Stations in 2019
We may earn a commission for purchases using our links. Learn more.


Navigation
Grassroots Infrastructure
Menu
Services
Docs
About
Blog
Contact
Search
content
<< Back to all services list
Simple options
Advanced options
Download job format
Copied to clipboard
Plot details
×
x
Success!
Added one set.


--- layout: default title: API Documetation redirect_from: "/api/" ---
The Redlink API provides a set of RESTful web services, grouped according their purpose: content analysis, semantic search and linked data publishing.
The Redlink Platform consists of a number of services that are offered to users through a REST API. The Platform provides three main services: Redlink Content Analysis, Redlink Linked Data Publishing and Redlink Semantic Search.
The Redlink Platform is implemented as "software as a service" (SaaS) running on a distributed cloud infrastructure. Users can access the system here through a uniform web service API using a private key to identify themselves. For the most common web frameworks, we will provide plugins providing out-of-the-box services. All requests to the API are handled by a single gateway which takes care of accounting and forwards the requests to backend clusters providing the necessary functionality.
You can register at my.redlink.io for getting free access to the platform for a trial period of 42 days (pricing details).
The Redlink API is organized in three major components:
| component | description | available |
|---|---|---|
| Content Analysis | Semantic Content Analysis, Interlinking and Enrichement | |
| Linked Data | Publish and query Linked Data, huge public datasets (DBPedia, Freebase, and many more) as well as your personal. | |
| Semantic Search | Combine the two above to turn search into find! |
All endpoints are available under the following schema:
https://api.redlink.io/1.0/{ENDPOINT}?key={KEY}&...
where:
Note: Take into account that all requests to all endpoints to the API require the key parameter. Therefore it'll be not mentioned further on their concrete documentations.
You can play around with our API console.
The API provides a special endpoint at the root:
https://api.redlink.io/1.0?key={KEY}
which supply some information about the usage of each app for the currently billing period (month):
{ "accessible": true, //provided key is valid and enabled "limit": 10000, //request limit/credit (within the current reporting period) "requests": 521, //requests sent/credit used so far "bytes": 345657, //bytes sent so far "seconds": 42 //total processing time "owner": "007", //user-id the key belongs to "analyses": [ //analysis services enabled for this key "my-analysis" ], "datasets": [ //datasets enabled for this key "my-dataset1", "my-dataset2" ] }
Keeping your data safe is important for us here at redlink, that's why the redlink API is only available using the https protocol. Therefore clients need to deal with TLS/SSL.
All SDKs for this API provided by us come with TLS/SSL support enabled. You should not trust clients that do not use the certificate to access the API.
Do you have any issue or question about the API? Please, contact us.
Redlink Content Analysis offers fact extraction, topic classification, and fact linking from textual and media documents in different languages. Users will be able to send text (and in the future media content) and get back facts and entities that have been identified automatically. In addition, the service will provide references to existing datasets (public datasets like Dbpedia and Freebase, as well as custom user-created vocabularies like a product database) about the entities identified in the content. Users will be able to create their own custom configurations with some simple steps, e.g. for adding advanced natural language processing features or commercial datasets.
Before using an analysis endpoint, the service must be configured in my.redlink.io and assigned to an application.
post /analysis/{analysis}/enhance
Performs an analysis of the content posted in the request body, getting enhancements results in the requested format. The {analysis} in the path bounds to the analysis to a concrete service previously configured.
Parameters
| name | description | required | restricted value/s |
|---|---|---|---|
| in | Format used for encode the content to analyze | optional (when missing, different heuristics are applied) | text, pdf, html, office |
| out | Format preferred to get the enhacement results | optional (when missing, HTTP content negotiation is applied) | json, xml, jsonld, rdfxml, rdfjson turtle nt |
| confidence | Set the minimum threshold for the confidence | optional, requires the usage of the Confidence Filter, where the default value is 0.85 | double (0.00..1.00) |
| summary | Request content summary | optional | true/false |
| thumbnail | Request Thumbnail URLs | optional | true/false |
The response could come in RDF (for jsonld, rdfxml and turtle formats) using this structure; or using a simplied version (for json and xml formats) which follows the following schema:
{ "languages": [ // Detected languages "en" ], "entities": [ // if no entity-extraction was done (e.g. because not activated in the analysis) // this property is completely omitted. { "dataset": "dbpedia", // The configured dataset (id/name/...) this entity comes from "reference": "http://dbpedia.org/resource/Saturday_Night_Live", // this is the URI of the entity "confidence": 1.0, // the maximum confidence of all mentions "properties": { // (1) basic properties, for every entity, always present "name": "name", "summary": "short summary", "type": [ // list of types, used to know which specific props are available (see below) "creative work", "event", "organisation", "person", "place", "product", "other" ], "sameAs": [ // sameAs links of the "reference" "http://rdf.freebase.com/ns/m.xxxxx", "http://yago-knowledge.org/resource/xxxx" ], // "image": [ (2) optional basic properties, might come with every entity ], "homepage": "http://example.org/", // optional // (3) type specific properties // (3.1) creative work "creator": "...", "genre": "...", // "publisher": "...", "datePublished": "2015-01-01", // (3.2) event "startDate": "2014-08-26T10:00:00Z", "endDate": "2014-08-26T12:30:00Z", // optional // (3.3) organisation /* nothing extra here */ // (3.4) person "gender": "male", "birth": "1974-04-01", "death": "2013-11-09", // optional ;-) "nationality": "de", // (3.5) place /* nothing extra here */ // (3.6) product "brand": "Apple", "manufacturer": "Foxconn", // (3.7) other /* nothing extra here */ // (3.8) Shared // (3.8.1) place || event || organisation (headquarter) "location": { "lat": 43.3333, "lon": 13.3333 } }, /* END properties */ "mentions": [ { "prefix": "c died on ", // 10 chars before the text "text": "Saturday", // aka anchor "suffix": " at the ag", // 10 chars after the text "context": " the context where the entity was recognized ", "confidence": 0.9, "start": 104, "end": 112 }, { /* more mentions ... */ } ] /* END mentions */ }, /* END entity */ { /* more entities ... */ } ], /* END entities */ "facts": [ // if no fact-extraction was done (e.g. because not activated in the analysis) // this property is completely omitted. { "name": "Product Weight", // preferred label of the fact-def. in the lang of the text "reference": "http://example.com/foo/ProductWeight", // URI of the fact-definition "type": "numeric", // enum, entity, numeric, range, dimension, date, duration, event "value": [ "foo", "bar" ], "mentions": [ { "prefix": "x50x15 mm ", "text": "Weight: 0.3 kg", "suffix": " Shipping ", "context": "...", "start": 15, "end": 26 }, { /* more mentions ... */} ] }, { /* more facts */ } ], /* END facts */ "categories": [ // if no classification was done (e.g. because not activated in the analysis) // this property is completely omitted. { "dataset": "dbpedia", "component": "machine-linking", "reference": "http://dbpedia.org/resource/xxxx", "confidence": 0.1234563, "properties": { "name": "...", "summary": "...", "image": [ "http://..." ] } }, { /* more categories */ } ], /* END categories */ "sentiments": { // if no sentiment analysis was done (e.g. because not activated in the analysis) // this property is completely omitted. "positive": 0.6213, "negative": 0.2112 } /* END sentiments */ "keywords": [ // if no keyword extraction was done (e.g. because not activated in the analysis) // this property is completely omitted. { "count": 2, //how often the keyword appears in the processed text. For n-grams this also //includes counts of sub-mentions. "keyword": "Hagia Sophia", //the extracted keyword "metric": 0.8167006795638168 // measure of the importance of the keyword // relative to the parsed text }, { /* more keywords */ } ] /* END categories */ }
Redlink Linked Data Publishing offers data management, data publication, and data integration for enterprise data using open standards and technologies (RDF and Linked Data); legacy proprietary data can be transformed into a standardized data model and integrated with data from other sources. The service will allow users to create their own Linked Data servers for managing vocabularies or publishing their own datasets (e.g. in Open Data projects). Datasets managed in this way can also be made available for content analysis and linking, and they can be either private (Linked Enterprise Data) or public (Linked Open Data).
Remember to enable your favourite datasets for the application (key) you are using!
Endpoints which have the dataset as part of the address restrict the operation to that dataset. Endpoints without a dataset in the path operate on all datasets configured for this application.
get /data/{dataset}
Export the dataset to the RDF serialization requested (using http content negotiation).
post /data/{dataset}
Import the raw RDF sent in the request body into the current dataset.
put /data/{dataset}
Import the raw RDF sent in the request body replacing the current dataset (i.e., removing all data before importing).
delete /data/{dataset}
Remove all data from the dataset (but it does not remove the dataset itself).
get /data/{dataset}/resource
Get a concrete resource in the RDF serialization requested (http content negotiation).
Parameters
| name | description | required | restricted value/s |
|---|---|---|---|
| uri | Resource URI | mandatory | valid URI |
post /data/{dataset}/resource
Add the raw data if the request body to this concrete resource. All triples with different subject will be ignored. Format is detected using content negotiation.
Parameters
| name | description | required | restricted value/s |
|---|---|---|---|
| uri | Resource URI | mandatory | valid URI |
put /data/{dataset}/resource
Replace the current resource with the raw data in the request body. All triples with different subject will be ignored. Format is detected using content negotiation.
Parameters
| name | description | required | restricted value/s |
|---|---|---|---|
| uri | Resource URI | mandatory | valid URI |
delete /data/{dataset}/resource
Remove a concrete resource from the dataset.
Parameters
| name | description | required | restricted value/s |
|---|---|---|---|
| uri | Resource URI | mandatory | valid URI |
get /data/resource
Get a concrete resource from all datasets in the RDF serialization requested (http content negotiation).
Parameters
| name | description | required | restricted value/s |
|---|---|---|---|
| uri | Resource URI | mandatory | valid URI |
get /data/{dataset}/sparql/select
Evaluates a SPARQL SELECT or ASK query over a dataset.
Parameters
| name | description | required | restricted value/s |
|---|---|---|---|
| query | Query string | mandatory | n/a |
| in | Format used to send data in the request | optional (when missing, regular HTTP mechanisms are applied) | |
| out | Format preferred to get the results | optional (when missing, HTTP content negotiation is applied) | json, xml, tabs, csv |
post /data/{dataset}/sparql/select
Evaluates a SPARQL SELECT or ASK query over a dataset.
Parameters
| name | description | required | restricted value/s |
|---|---|---|---|
| query | Query string | optional (if missing, a application/x-www-form-urlencoded should be sent and query will be read from the request body) | n/a |
| in | Format used to send data in the request | optional (when missing, regular HTTP mechanisms are applied) | |
| out | Format preferred to get the results | optional (when missing, HTTP content negotiation is applied) | json, xml, tabs, csv |
get /data/{dataset}/sparql/update
Evaluates a SPARQL UPDATE query over a dataset.
Parameters
| name | description | required | restricted value/s |
|---|---|---|---|
| update | Query string. | mandatory | n/a |
| in | Format used to send data in the request | optional (when missing, regular HTTP mechanisms are applied) | |
| out | Format preferred to get the results | optional (when missing, HTTP content negotiation is applied) | json, xml, tabs, csv |
post /data/{dataset}/sparql/update
Evaluates a SPARQL UPDATE query on the data.
Parameters
| name | description | required | restricted value/s |
|---|---|---|---|
| update | Query string. When missing a application/x-www-form-urlencoded should be sent and query qould be ready from the request body. | optional | n/a |
| in | Format used to send data in the request | optional (when missing, regular HTTP mechanisms are applied) | |
| out | Format preferred to get the results | optional (when missing, HTTP content negotiation is applied) | json, xml, tabs, csv |
get /data/sparql
Evaluates a SPARQL SELECT or ASK query over all datasets.
Parameters
| name | description | required | restricted value/s |
|---|---|---|---|
| query | Query string | mandatory | n/a |
| in | Format used to send data in the request | optional (when missing, regular HTTP mechanisms are applied) | |
| out | Format preferred to get the results | optional (when missing, HTTP content negotiation is applied) | json, xml, tabs, csv |
post /data/sparql
Evaluates a SPARQL SELECT or ASK query over all datasets.
Parameters
| name | description | required | restricted value/s |
|---|---|---|---|
| query | Query string | optional (hen missing, a application/x-www-form-urlencoded should be sent and query will be read from the request body) | n/a |
| in | Format used to send data in the request | optional (when missing, regular HTTP mechanisms are applied) | |
| out | Format preferred to get the results | optional (when missing, HTTP content negotiation is applied) | json, xml, tabs, csv |
get /data/{dataset}/ldpath
Evaluates a LDPath program over a dataset.
Parameters
| name | description | required | restricted value/s |
|---|---|---|---|
| program | Program string | mandatory | n/a |
| uri | Resource where to start to evaluate the program | mandatory | valid URI |
post /data/{dataset}/ldpath
Evaluates a LDPath program over a dataset.
Parameters
| name | description | required | restricted value/s |
|---|---|---|---|
| program | Program string | mandatory | n/a |
| uri | Resource where to start to evaluate the program | mandatory | valid URI |
get /data/ldpath
Evaluates a LDPath program over all datasets.
Parameters
| name | description | required | restricted value/s |
|---|---|---|---|
| program | Program string | mandatory | n/a |
| uri | Resource where to start to evaluate the program | mandatory | valid URI |
post /data/ldpath
Evaluates a LDPath program over all datasets.
Parameters
| name | description | required | restricted value/s |
|---|---|---|---|
| program | Program string | mandatory | n/a |
| uri | Resource where to start to evaluate the program | mandatory | valid URI |
get /data/{dataset}/release
Releases the current data in the dataset for being use in the analysis.
post /data/{dataset}/release
Releases the current data in the dataset for being use in the analysis.
In addition, all resources in a dataset explicitly published by the user will be publicly available according the Linked Data principles through our data hub (data.redlink.io) under the following schema:
http://data.redlink.io/{USER}/{DATASET}/...
Redlink Semantic Search offers high performance faceted and semantic search over enterprise content, different ranking algorithms, auto completion, thesaurus/vocabulary integration, etc.; the service will provide advanced search functionalities based on Content Analysis and Linked Data. Users will be able to create their own search configurations, upload documents, analyse and link them with datasets and vocabularies, and provide powerful search functionalities, e.g. browsing documents by (automatically detected) categories, persons occurring in the text, etc.
Semantic Search is not yet part of the public API, only available for premium customers; but it will be included in the next versions, so stay tuned... In the meantime, you can check either our Search Engine or use the Redlink Solr plugin to semantically enrich your current search.


--- layout: post title: An apology to Ohanian (Reddit Co-founder) date: 2010-09-01 17:39:00.000000000 -05:00 comments: true categories: [] published: true ---
In a prior post I came out in swinging defense of what I thought was Kevin’s courageous attempt to save Digg and Alexis’s misguided criticisms. As it turns out, Digg V4 has decided to focus on ‘large publishers’ (like Alexis had thought they would) rather than ‘small publishers’ (whichh I thought they were). So, given that I quickly (and loudly) appealed to Alexis to be cautious with his critique, I feel it is appropriate to (just as loudly) apologize and admit that I was wrong, and am now disappointed in Digg and Kevin. I stand corrected Alexis, you were right. Please note that I am not apologizing just because the community revolted – communities always revolt when change happens, but they often get used to it – but I am acknowledging that Alexis correctly saw what the true changes at Digg were, as I did not.


Project Description
This was a year-long capstone group project that was presented for the 2021 USC Games Expo.
I worked as the usability lead, recruiting playtesters and collecting feedback for each iterative build. After each session, I then organized the raw data into weekly reports directing the team on what the game was lacking in and where future efforts should be focused. Additional duties included periodic program-wide presentations on the state of the game and plans up until official release.
A Closer Look
You can download the game here.
Tools Used
Google Sheets
I used Google Sheets to organize all collected data into a single spreadsheet that could be accessed and easily read by the team. I followed the guidelines of RITE testing, which meant the rapid testing of successive builds against fresh testers, documenting problems encountered in each session, and classifying them by their degree of severity (failure, error, miss, complaint).
Additionally, documenting the source and situation of an issue was done in order to provide context for the design team and expedite their process of finding a solution. Over time, the trend of problems ideally declined in both number of problems identified as well as the degree of severity.
Zoom
Zoom was used for its convenience of setting up playtest sessions and recording functionality. A standard testing session included a brief introduction and briefing of expectations, any necessary pre-playtest troubleshooting, a blind playthrough of the build provided, and then debriefing followed by an interview and questionnaire.
Google Forms
Following the interview component of a playtest, Google Forms was employed to collect written feedback for easier data presentation. Metrics collected from the forms as well as built into the game itself can be then displayed in visual mediums like charts for ease of presentation when reporting back to the team or a larger audience.


DMPlanner
Create your DMP in just a few steps with the help of DMPlanner!
Name
Please provide your full name.
Search ORCiD
Resources
Add as many Github repositories or OAI-PMH compliant DOIs as you like.
*
Add Resource
Preservation Time
Choose how many years the data for each group should be kept.

Generate DMP


FOP: Mailing List Resources
General Information
Before posting questions to any list:
- Review the Jakarta Mailing Lists - Guidelines for general principles about mailing lists.
- Review "How To Ask Questions The Smart Way".
- Send only plain text email messages to any mailing lists. Please do not send html or rtf email, as they do not work well with the archive engines. If you are using Microsoft Outlook, this setting can be found at the "Mail Format" tab of the Tools/Options menu.
- Use a mail client that respects the References and/or In-Reply-To mail header entries. This is important for mail clients that can display threads as trees. Compliant mail clients include Microsoft Outlook, Becky!, KMail, Apple Mail, Ximian Evolution, Mozilla (version 1.2 or greater). Non-compliant mail clients include Microsoft Exchange, Lotus Notes, Internet Mail Service (MUA or MTA), and dtmail.
For help in understanding email acronyms, see the Lingo2Word Acronym List, or the Keno Internet Services Internet Glossary.
FOP Users Mailing List
Use this forum to discuss topics of interest to FOP users.
Archives
To review the archives, you have several options:
- The Apache Mailing List archive (mod_mbox archive, no full-text search, yet).
- The Apache Mailing List archive (gzipped mbox files).
- The Mailing list ARChives (MARC) at the AIMS group.
- The Mail Archive.
- The GMANE archive.
- The Nabble archive (only posts after May 2005).
- The MarkMail archive.
Subscription Information
- See Apache XML Graphics Mailing Lists for detailed subscription information.
- To subscribe (digest only): Send email to<EMAIL_ADDRESS>- To subscribe fully: Send email to<EMAIL_ADDRESS>- To receive information about the mailing list: Send email to<EMAIL_ADDRESS>- To unsubscribe: Send email to<EMAIL_ADDRESS>Submitting a Question
FOP support is primarily self-service. The FOP User Mailing List serves as a backup to the self-service documentation for cases where either the documentation is deficient or where users have unusual circumstances. FOP developers and users are happy to help answer questions that are appropriate to the forum (i.e. FOP-specific), and that are submitted after appropriate preparation. To ensure that your question is not abusive of this policy, please use the following checklist:
- Have you followed the Getting Help checklist? If not, please do so before submitting your question.
- Is your question appropriate to the forum? If it is really an XSL-FO question, XSLT question, or PDF question, please see the other resources on this page that are intended to help you get those questions answered.
- Have you read Mailing List General Information? If not please do so before proceeding.
- Have you stated the version of FOP you are using? Please do so. Usually, it's a good idea to state the JDK/JRE version and the operating system you're using, too.
- Have you included any detailed error messages? Please do so.
- Does a proper understanding of your question require inclusion of XSLT code, DocBook source, or other semantic XML? If so, the question is almost certainly not appropriate to this list. In general, the only input documents that are appropriate on this list are XSL-FO snippets. See Running Xalan for instructions about capturing the XSL-FO document that is actually submitted to FOP. If you haven't examined the XSL-FO document yourself, then you are not yet prepared to formulate a FOP-specific question.
- If you are providing one or more XSL-FO snippets:
- Have you reduced them to the shortest possible complete, self-contained document that demonstrates the problem? Please do so.
- Have you removed images that are not an integral part of the question? Please do so.
- Have you filtered out confidential material? Please do so.
- If you are including a stack trace:
- Is it helpful in finding the problem? If not, please do not submit it.
- Have you included only those portions that are relevant to the question? If not, please do so.
- Are you attaching large PDF files or screen shots to your message? If so, please consider attaching a minimal, appropriate B&W GIF, JPG or PNG that conveys the necessary information instead. A good alternative is to put the files on a web server.
XSL-FO Mailing List (at W3C)
Use this forum to ask general XSL-FO questions.
- To review the archive: W3C XSL-FO Archives.
- Before posting questions to any list, see "General Information".
- Subscription administration information can be found at W3C Mailing List Administrativia. After reviewing the instructions there, send your subscribe or unsubscribe request to<EMAIL_ADDRESS>XSL-FO Mailing List (at YahooGroups)
Use this forum to ask general XSL-FO questions.
- Before posting questions to any list, see "General Information".
- The home page for this groups is XSL-FO - discussion of XSL Formatting Objects.
XSLT List (Mulberry Tech)
- Before posting questions to any list, see "General Information".
- Information for using and subscribing can be found at XSL-List -- Open Forum on XSL.


Change first name from {{ user_data[0] }} to
Change first name from {{ user_data[1] }} to
Your current activity is set to {{ user_data[3] }}
Pick activity
Are you sure you want remove your membership


--- layout: post title: FP titbits date: 2008-09-30 00:00:00 categories: business mobile personal software-development ---
It's been a good week for a few FP clients past and present, and seeing as FP is a hefty chunk of my life, you get to read about it here ;)


More GReader hacking — immersion 2008/02/11
First, some updates to readelicious. I worked in Mike Malone's patch to add a keyboard shortcut which I now can't live without. I also reworked some bits of it to interact nicely with the new script.
Immersion is another Greasemonkey script that will load the current entry as an <iframe> within Google Reader. I made this one almost exclusively to look at A Brief Message but it works just as well for leaving a quick comment since it doesn't interrupt your train of thought like opening a new tab does (I tend to forget new tabs until hours later).
Just like readelicious, immersion adds a link to each entry but also uses 'i' for keyboard access to the current entry.
http://svn.rcrowley.org/svn/greasemonkey/immersion.user.js or http://userscripts.org/scripts/show/22685
Comments (2)
hey is there anyway to add a background to the script, i love the transparency effect but its hard to read, a nice white background or 50% grey would be really nice
-r
— rahim — 2008/02/19 3:35 pm
Uh I have a complaint, SIR. If I am "immersed" in something and hit 'i' again, instead of disappearing, it just adds another immersion window. Where are your UI chops, sir?
— Mike Panchenko — 2008/03/25 5:33 am


--- layout: default title: {{ site.name }} ---
Sinap is not a plot IDE. The goal of Sinap is to provide a generic UI platform for editing and interpreting domain-specific graph-based languages. What does this mean? Let's break it down. Users of Sinap want to design visual representations of state machines or other conceptual tools that can be modeled with nodes and edges but don't want to have to create their own UI to handle all the fancy drawing stuff. Sinap IDE will provide the heavy lifting for presenting the visual side of creating graphs as well as running interpreters for them. Where do these interpreters come from? Plugins. Sinap IDE will expose an API that developers can use to create their own domain-specific interpreters of languages that can be modeled with graphs. These plugins can then be installed into Sinap, giving the IDE more power.
There are many options for graph editors available, but the vast majority of them are designed specifically for their respective domains. Sinap IDE aims to break this trend by allowing developers implement their own interpreters that can be used in the application. Think of a lightweight text editor for writing code but replace "text" with "graph" and add interpreting and that is what Sinap aims to be. What do we mean by "interpret"? Sinap is not a tool for simple chart designs. The idea behind Sinap plugins is to be able to feed graphs some input in order to see what the graph produces. Imagine a tool that allows you to step through a state machine that you designed for a project. With Sinap this is possible.


Joomla Users ajax importer from CSV file. Fill the form below and import new users into joomla database.
version 3.0.0
Remove this software
For security reasons, remember to remove this sofware
from your server when users are imported!


after the stack of decorators has been defined you may need to access decorators. DecoratorVisitors allow you to do this.


Noticias
Amoung Us
It is a long established fact that a reader will be distracted by the readable content of a page when looking at
its layout. The point of using Lorem Ipsum is that it has a more-or-less normal distribution of letters, as
opposed to using 'Content here,
Publicado pelo administrador
Call of duty: Mobile
It is a long established fact that a reader will be distracted by the readable content of a page when looking at
its layout. The point of using Lorem Ipsum is that it has a more-or-less normal distribution of letters, as
opposed to using 'Content here,
Publicado pelo administrador


I am Junior Web Developer from Upstate New York.
Executed manual testing. Working collaberatively with development team in anticipation of 12/18 website re-launch. Wrote test cases for new website features.
Prospected and opened new customers for unit pride merchandise, specifically targeting the U.S. Army.
Organized and delivered full postal mail routes in variety of zip codes.
Worked closely with IT to identify bugs during website launch. These included delayed or lost orders, incorrect or failed import of customer shipping and billing info. Collaborated with buyers to coordinate purchase orders tied into demands of key accounts.
Beat reporter for high school sports, local politics with Print circulation of 12,000.
I love fresh air and activities such as biking, hiking and surfing when I'm near a coast with waves! Having a great appreciation for music (especially blues), I have taken up to learning the bass guitar. Technology fascinates me, including learning about it's role in our lives and how we can harness it to make a better future for all. I also love travel, especially when the focus is on historical places and sites.


Report
Response Time timelines
Inner state variables timeline
Inner State Variable against Response Time
Measured Response Time against Predicted Response Time by Neural Network (set All)
Plot measured Response Time and predicted Response Time by Neural Network normalized (set All)
Plot measured Response Time and predicted Response Time by Neural Network de-normalized (set All)
What would be the metrics values after scaling vs Real values
Estimated Response Time if add different numbers of resources
Response Time and proposed number of additional resources on Test Set
Predicted Response Time and proposed number of additional resources on Test Set


{{ feature.text }}
{{ feature.description }}
--- title: Pricing heading: Our subscription model description: Contact us for more information on our offering. pricing_table: - name: Platform price: €950,- per month features: - text: HyperDev Dashboard description: | The HyperDev Dashboard exposes a Graphical User Interface and API for managing application definitions (docker-compose+), services (Docker Stacks) and storage (buckets). The dashboard is aimed at software engineers and QA (testing + reporting). Consequently, it is easy to use and supports all CI/CD preconditions. - text: Compute Cluster description: | The compute cluster is a Docker Swarm. You will get your own cluster of multiple nodes with a total of 48GB memory and 12 processing cores. It is used for running all CI/CD services such as Jenkins, SonarQube, Docker Registry, as well as your own applications. It is backed by the Persisted Storage. Because of this setup, it is easy to extend the cluster with extra memory and cores. Applications can safely run next to each other, using smart memory limits. - text: Dedicated Jenkins Build Server description: | In order to build software and make use of Jenkins CI pipelines, it is essential to have a dedicated build server. You will get full access to this build server (via SSH) and be able to run any build scenario. The build server has 4GB of memory and will run on 2 cores. The operating system we use is CentOS 7. - text: Persisted Storage description: | The persisted storage will store all data buckets for HyperDev and has initial capacity of 200GB. One can, of course, always purchase more. The persisted storage runs NFS on LVM so it is easy to scale up the storage without any downtime. In order to keep control of the data created through the CI/CD pipelines, the platform will periodically run cleanup scripts. - text: Secure VPN description: | Security is an essential part of any platform. That is why we configure and run an OpenVPN server to allow you to connect to services running on HyperDev. Users can run an OpenVPN client to connect and make sure their data remains safe. - text: Development Environment description: | The development environment consists of the HyperDev Dashboard and the Compute Cluster. Together they create what we like to call a 'Continuous destroyment' setup. This allows users to create new instances of different versions of the application you are developing as throw-aways. It's as easy as start (an instance) -> check (run tests against it) -> stop (free up resources). No more dedicated Development and Test environments with mismatching configurations, maintained by expensive IT experts. Leverage the potential of containerization in your development process! - text: CI/CD Environment description: | In order to actually have proper CI/CD pipelines, you need the right tools. All the tools required to run our example application and setup will be provided to you - no more struggling for weeks until everything is connected! Start delivering customer value from day one! - text: Backup description: | All data in the persisted storage is backed up every 4 hours and is kept at least 2 days. There is also a weekly backup available for up to 5 weeks. Data can be recovered at your request. - text: User Accounts description: | You will get access to the platform (VPN and LDAP accounts) for up to 10 users. You have a big project and need more users? No worries, more user accounts can be purchased at any time. - text: Technical support description: | The HyperDev team is ready to help you with all technical issues. We have a service desk that is available during office hours (8:00 - 20:00) for platform related questions and incidents. call_to_action: link: /contact/ text: Contact us - name: Consultancy price: Starting at €2000,- per month features: - text: The Platform package description: Of course everything from the platform package is included. You just pay the regular base price with added consultancy hours. - text: Custom integrations description: Together with our consultants we help you to integrate existing systems with HyperDev. Both on-premise and cloud services can easily be integrated. - text: Consultancy and support description: Our consultants help you setup, configure and integrate HyperDev in your company. They provide first line support. - text: Training description: Our consultants are available to provide you with expert trainings regarding our platform or DevOps practices in general. - text: Provided by our partners description: The consultancy package is currently provided by our implementation partners. call_to_action: link: /contact/ text: Contact us addon_table: - name: 'Extra Cluster Resources' color: "#4a4a4a" features: - text: €280,- per month highlight: true - text: 16GB memory, 4 cores highlight: false - text: Extra compute resources for the HyperDev cluster highlight: false - name: Extra Persisted Storage color: "#4a4a4a" features: - text: €55,- per month highlight: true - text: 100GB highlight: false - text: Adds extra storage to the HyperDev cluster highlight: false - name: Extra Jenkins Build Machine color: "#4a4a4a" features: - text: €115,- per month highlight: true - text: 4GB memory, 2 cores highlight: false - text: Dedicated build node for Jenkins highlight: false - name: Extra Users color: "#4a4a4a" features: - text: €200,- per month highlight: true - text: Access for 10 extra users highlight: false - text: highlight: false faq: - question: What is included in the default development environment setup? answer: The default CI/CD setup includes the following preconfigured tools; Gitlab, Jenkins, SonarQube, Docker Registry, Example Application with Tests and an Example continuous delivery pipeline. - question: Can I buy additional memory or computing power? answer: Yes, we offer additional cluster nodes to extend your environment. See above for pricing. - question: Can I buy additional storage? answer: Yes, we offer additional persisted storage to extend your environment. See above for pricing. - question: Can I integrate my existing cloud or on-premise tools/services? answer: Yes, we offer consultancy through our implementation partners. Contact us for more information. - question: Can I add additional user accounts? answer: Yes, the default package includes up to 10 user accounts. Extra users can be added to your environment. See above for pricing. - question: Questions? answer: Contact us for any further questions at<EMAIL_ADDRESS>---
{{ feature.description }}
HyperDev Addons


THE BRIDE & GROOM
WEDDING
ACCOMODATIONS
REGISTRY
RSVP
RSVP
NOVEMBER 29
th
, 2014
Name
Choice of Dish:
Steak
Fish
Name
Choice of Dish:
Steak
Fish
Total No. of Guests
1
2
3
4
5
Comments or Special Requests:


--- layout: default title: Worldwide React Redux courses description: React Redux courses worldwide. Small groups and project-based learning. menu: next-courses redirect_from: /react-redux-training url: /react-redux-graphql-training permalink: /react-redux-graphql-training ---


|
||||||||
| PREV NEXT | FRAMES NO FRAMES |
Block
objects, links them together, and knows how to verify that the
chain follows the rules of the NetworkParameters
for this chain.Utils.doubleDigest(byte[],int,int)
.
StoredBlock
that represents the top of the chain of greatest total work.
Object.hashCode()
.
StoredBlock
s in memory.Peer
.
StoredBlock
that represents the top of the chain of greatest total work.
Block
object with extra data that can be derived from the block chain but is slow or inconvenient to
calculate.
|
||||||||
| PREV NEXT | FRAMES NO FRAMES |


Before Change
def __init__(self, inplanes, planes, stride=1, downsample=None, noactivation=False):
super(RevBottleneck, self).__init__()
if downsample is None and stride == 1:
Gm = BottleneckSub(inplanes // 2, planes // 2, stride, noactivation)
Fm = BottleneckSub(inplanes // 2, planes // 2, stride, noactivation)
self.revblock = ReversibleBlock(Gm, Fm)
else:
self.bottleneck_sub = BottleneckSub(inplanes, planes, stride, noactivation)
self.downsample = downsample
self.stride = stride
def forward(self, x):
After Change
self.conv1 = conv1x1(
in_channels=in_channels,
out_channels=mid_channels)
self.conv2 = pre_conv3x3_block(
in_channels=mid_channels,
out_channels=mid_channels,
stride=stride)
self.conv3 = pre_conv1x1_block(
in_channels=mid_channels,
out_channels=out_channels)


About
Created by Remy Sharp as a child of the late 70s the ZX Spectrum are his fondest memories of computing as a kid. So it only made sense that he turn his browser-based skills to made things work on the Spectrum.
Inspired by the return of Spectrum via the ZX Next, Remy learnt how loading screens worked, how the binary data in screens is arranged and presented his work in a mad demo on stage in 2018.
Open Source
The code is a little hacky (written over a few days), but has some semblance of organisation. It can be found in github and is available under the MIT license:
https://github.com/remy/zx-tools
Seen an issue or want to make these tools better? Raise an issue
Text 2 BASIC
Features:
- Drag and drop support
- Can upload and parse 3DOS+ .bin files and plain .txt
- Export as:
- .TAP
- .bin 3DOS+ format - compatible with ZX Spectrum Next
- .wav with correct headers and pilot for tape loading
- Import from gists, github and gitlab via query string, such
as:
?id=3cbe71821d3eae96273d0cef0426fda0 or
?url=https://gitlab.com/thesmog358/tbblue/-/blob/master/games/Next/NextBASIC%20Invaders/basicInvaders.bas
- Parses
10print"ok"
and correctly expands - Unique line numbers, so repeating a line number will replace previous line
- Session remembers last code
Sprites
Editor based on the .spredit
tool on the ZX Spectrum Next.
Features:
- Exports to
.spr
ZX Spectrum Next compatible format - Supports both 16x16 and 8x8 tilemaps
- Drag and drop import
- Can import spectrum binary font files and map to sprites
- Palette editor with priority support
- Imports PNG file format for easy sprite creation
- Import PNG to sprite sheet support dimensions at multiples of 16x16 blocks - i.e. 32x48 or 16x256 etc
- See usage panel for full list of keyboard and mouse shortcuts
Tools
The general tools allow you inspect the contents of TZX and TAP files, and extract and inspect sections, with support for font, SCREEN$ and some BASIC previewing.
The tools can also take any JPG image (in any resolution) and convert it to a ZX Spectrum compatible 15 colour image that be rendered into Spectrum memory.
There is also support for TTF font importing which approximates font rendering for the spectrum.


Pages
About
Contact
Pricing
Services
Sign in
Sign up
Support
Documentation
v3.0
Examples and guides
Support
Found an issue? Let us know!
Upgrade to Pro
Docs v3.0
微信公众号
PokeScript
by 野生的妙蛙种子
Help


Macklin Lee's Vacation to
Place
Destination
Description
Cost
Garden
Much Pretty
$1
Fountain
So water
$2
Bamboo
Very tall
$3
Trees
Wow yellow
$4


WITNESS LIST Revenue & Public Education Funding, Select Committee December 1, 1997-10:00A Texas Economy On: Allaway, Bill (Tx Taxpayers & Rsch Asc) Hamilton, Billy (Comptroller's Office) Hazleton, Jared (himself, TAMU Economist) Howden, Robert (NFIB/Texas) James, Reggie (Consumer's Union) Lavine, Richard (Cntr. Pub. Polcy. Prior) Perryman, M. Ray (The Perryman Group) Reissig, Mike (Comptroller's Office)


art
home
about us
page
shop
blog
contact us
affordable paris tours
city tours / tour tickets / tour guides
new collection
blue thirt
$26
like
repost
time
add to cart
woman shirt
$31
like
repost
time
add to cart
yellow tshirt
$21
like
repost
time
add to cart
cool lingerie
$56
like
repost
time
add to cart


Introduction
ParamountDialogs
is a delightful dialog view for iOS in Swift.
Requirements
- iOS 8.0+
- Xcode 9 with Swift 4
Installation
CocoaPods
use_frameworks!
pod 'ParamountDialogs'
ParamountDialogs Docs (43% documented)
ParamountDialogs
is a delightful dialog view for iOS in Swift.
use_frameworks!
pod 'ParamountDialogs'


College Placement Organiser
Welcome to College Placement Organiser
Placement organizer helps for on-campus and off-campus recruitment.
TPO addds recruitment information and sends mail or SMS to selected students
students are benefited by prior informtion


{% extends "base.html" %} {% load i18n %} {% load static %} {% block head %} {{ block.super }} {% endblock %} {% block content %}
{{ job.description | safe }}
Are you sure you want to delete this job?
{{ job.company.name }}
{{ job.location }}
{{ job.date_posted }}
See the full listing here


I wasn’t able to attend useR! 2016 last week. But thanks to Microsoft’s live streaming service, I was able to record a number of sessions and watch them over the holiday weekend. I am a frequent user of many of Hadley Wickham’s R packages. So of course I tuned in for his keynote. Hadley presented the following map, using historical data from the USAboundaries
package. His point wasn’t about map making, per se – but rather about “tidy” principles for dealing with complex data structures (such as shape files or statistical models). I’m onboard with the #tidyverse. But I was also inspired by the map itself.
In honor of US Independence Day, I decided to build on Hadley’s example to make an animated map of US settlement. I don’t have many occasions to make maps as part of my current academic work. So I thought it would be fun to spend some time on the 4th of July learning to wrangle shape files in R.
Goals
My primary goal was to make two maps:
- an animated map showing the settlement of US territories and the conversion of territories to states; and
- an animated map showing the development of US counties.
Secondly, I wanted to focus the maps on the continental US, while still showing data from all regions. Repositioning / reprojecting Alaska and Hawaii is certainly not news. But for a newcomer to geospatial data, it was a data-munging benchmark that I needed to surpass.
Dataset
The USAboundaries
package contains state/territory-level data from September 3, 1783 to December 31, 2000. So the state-level map will be a bit truncated historically – skipping over the early settlement of US colonies, as well as the declaration of the 13 original colonies as free and independent states in 1776(!). But alas, we’ll still have a nice bit of data to play with.
The county-level data in the USAboundaries
package dates back about a century and a half further than the state-level data, all the way to December 30, 1636. So this dataset will provide a more extended view of US settlement history.
Maps
Here are the final results of my holiday weekend map-making expedition. Annotated R code to reproduce these maps is below.
The state-level map shows the United States on January 1st, every year from 1784 to 2000. (I grew up in Oklahoma – but I’d forgotten that there was a short period just before Oklahoma became a state in 1907 during which part of the region was an organized incorporated US territory and part was unorganized. This animated map reminded me of that fact).
The county-level map shows the United States on January 1st, every year from 1637 to 2000. One thing I like about this animation is that it highlights the degree of geographic separation during the early years of settlement.
R code
Data wrangling process
The heavy lifting is done by the get_boundaries()
function defined below. This function uses the USAboundaries::us_boundaries()
function to get (state- or county-level) shape files for a specified date, and then performs some additional processing to convert the shape file into a convenient format for plotting. The additional processing involves the following:
- extract metadata associated with each region (e.g., whether the region is an unorganized territory, state, etc.)
- project the coordinate system to Albers equal area
- reposition Alaska and Hawaii – I learned a lot about this from a helpful post by Bob Rudis, AKA @hrbrmstr.
- combine metadata with (repositioned) shape data
library(USAboundaries) # contains data (shape files)
library(dplyr) # for wrangling
library(lubridate) # for wrangling dates
library(rgdal) # for wrangling shape files / map data
library(maptools) # for wrangling shape files / map data
library(ggplot2) # for plotting
library(gganimate) # for gif animation
library(viridis) # for lovely color palettes
## ------------------------------------------------------------------------ ##
get_boundaries <- function(date, type = "state"){
# Get shapefile for US states on specified date
sp <- us_boundaries(as.Date(date), type = type)
# Extract metadata. Includes info about the territory type for
# each region (i.e., state, unorganized territory, etc.)
metadata <- sp %>%
as.data.frame() %>%
mutate(id = as.character(id_num -1))# %>%
# select(id, name, terr_type)
# ----------------------------------------------
# Convert coordinate system to Albers equal area,
# and then reposition Alaska and Hawaii.
#
# thanks to Bob Rudis (hrbrmstr):
# https://github.com/hrbrmstr/rd3albers
sp_t <- spTransform(sp, CRS("+proj=laea +lat_0=45 +lon_0=-100 +x_0=0 +y_0=0 +a=6370997 +b=6370997 +units=m +no_defs"))
rownames(sp_t@data) <- as.character(as.numeric(rownames(sp_t@data)) - 1)
# extract, then rotate, shrink & move alaska (and reset projection)
alaska <-
if(type == "state"){
sp_t[grep("Alaska", sp_t$full_name, fixed = TRUE),]
} else if (type == "county"){
sp_t[grep("Alaska", sp_t$state_terr, fixed = TRUE),]
}
alaska <- elide(alaska, rotate=-50)
alaska <- elide(alaska, scale=max(apply(bbox(alaska), 1, diff)) / 2.3)
alaska <- elide(alaska, shift=c(-2100000, -2500000))
proj4string(alaska) <- proj4string(sp_t)
# extract, then rotate & shift hawaii
hawaii <-
if(type == "state"){
sp_t[grep("Hawaii", sp_t$full_name, fixed = TRUE),]
} else if (type == "county"){
sp_t[grep("Hawaii", sp_t$state_terr, fixed = TRUE),]
}
hawaii <- elide(hawaii, rotate=-35)
hawaii <- elide(hawaii, shift=c(5400000, -1400000))
proj4string(hawaii) <- proj4string(sp_t)
# remove old states and put new ones back in
sp_t <-
if(type == "state"){
sp_t[!grepl("Alaska|Hawaii", sp_t$full_name),]
} else if(type == "county") {
sp_t[!grepl("Alaska|Hawaii", sp_t$state_terr),]
}
sp_t <- rbind(sp_t, alaska, hawaii, makeUniqueIDs = TRUE)
# ----------------------------------------------
# get borders for each region
borders <- sp_t %>%
fortify() %>%
as.data.frame() %>%
mutate(year = year(date)) %>%
left_join(metadata, by = "id")
}
The create_timeline()
function creates the data needed to draw an animated timeline on the graphs. The input to this function is a dataframe containing lat/long info about the map to be plotted (e.g., the dataframe created by a call to get_boundaries()
). The basic idea is to divide the x-axis range of the map (i.e., min and max longitude) into evenly spaced intervals, one for each year (since we’re animating by year). The resulting sequence provides the tick locations for the moving timeline tick point. And then we determine the vertical placement of the timeline by getting the max latitude of the map and adding a bit of padding.
create_timeline <- function(boundary_df){
# determine how many year's worth of data will be plotted
years <- unique(boundary_df$year)
# create sequence that spans the width of the map (from min to max longitude),
# with evenly spaced intervals for years
timeline_range <- with(boundary_df,
seq(from = min(long),
to = max(long),
length.out = length(years)))
# determine latitude coordinate for positioning timeline.
# (max latitude * 1.2 places timeline just above map)
timeline_y <- max(boundary_df$lat) * 1.2
# collect timeline data into a df
timeline <- data.frame(
x = timeline_range,
y = timeline_y,
year = years
)
return(timeline)
}
With these two helper functions in hand, we can now get to the business of making the maps.
State-level map
# specify dates for which historic map data is to be gathered
state_dates <- seq(as.Date("1784-01-01"), as.Date("2000-01-01"), by = "year")
# for faster rendering, try plotting a reduced date range
# state_dates <- seq(as.Date("1784-01-01"), as.Date("1789-01-01"), by = "year")
# get state-level data for every date, bind all data into one df,
# and set factor levels for territory type
states <- lapply(state_dates,
function(x) get_boundaries(x, type = "state")) %>%
do.call(rbind, .) %>%
mutate(
terr_type = factor(terr_type,
levels = c("State",
"District of Columbia",
"Territory",
"Unorganized Territory",
"Other")))
# create timeline for state data
state_timeline <- create_timeline(states)
# plot the map
p_state <- ggplot() +
# Add territory polygons
geom_polygon(aes(x = long, y = lat, group = group, fill = terr_type, frame = year),
data = states,
alpha = 0.8) +
# Add timeline
geom_segment(aes(x = min(x), xend = max(x),
y = y, yend = y,
frame = year),
data = state_timeline,
colour = "grey50", size = 1) +
geom_point(aes(x = x, y = y, frame = year),
data = state_timeline,
colour = "grey50", size = 3) +
# styling
coord_equal() +
scale_fill_viridis(discrete = TRUE) +
labs(title = "Map of US territories\nJanuary 1,",
fill = "") +
theme_minimal() +
theme(panel.grid = element_blank(),
axis.text = element_blank(),
axis.title = element_blank())
gg_animate(p_state, interval = 0.1)
County-level map
# specify dates for which historic map data is to be gathered
county_dates <- seq(as.Date("1637-01-01"), as.Date("2000-01-01"), by = "year")
# get county data for specified date range
counties <- lapply(county_dates,
function(x) get_boundaries(x, type = "county")) %>%
do.call(rbind, .)
# create timeline
county_timeline <- create_timeline(counties)
# make the plot
p_counties <- ggplot() +
# Add county polygons
geom_polygon(aes(x = long, y = lat, group = group, frame = year),
data = counties,
colour = "red", fill = "grey90", size = .1) +
# Add timeline
geom_segment(aes(x = min(x), xend = max(x),
y = y, yend = y,
frame = year),
data = county_timeline,
colour = "grey50", size = 1) +
geom_point(aes(x = x, y = y, frame = year),
data = county_timeline,
colour = "grey50", size = 3) +
# Styling
coord_equal() +
labs(title = "Map of US counties\nJanuary 1,") +
theme_minimal() +
theme(panel.grid = element_blank(),
axis.text = element_blank(),
axis.title = element_blank(),
plot.title = element_text(colour = "white"),
legend.text = element_text(colour = "white"),
plot.background = element_rect(fill = "black"))
# animate the plot
gg_animate(p_counties, interval = 0.1)
Final notes
The Rmd file used to create this report is available here.
Note that if you run this code on a standard laptop, it will likely take several (5-15) minutes to make each map due to the amount of data that is prepped + rendered. The county map is particularly slow because the data frame contains more than 11 million rows and the gif contains nearly 400 frames (one frame per year from 1636–2000).
My approach here (and in general) was desired output first, optimize later, especially since I’m new to map making. If you just want to play around with the code and see immediate results, I would start by reducing the date range to a handful of years, which will dramatically speed up the data prep + rendering process.
Also, I just learned about the new rmapshaper
package, which might be one way to improve rendering speed at scale. This package performs multi-polygon simplification by removing overlapping boundaries between adjacent polygons. Maybe I’ll try that out in a later post.
Comments are always welcome!
comments powered by Disqus


How to Install Georestricted Apps on iPhone
by Karanpreet SinghWell, if we talk about the best mobile operating system, without any doubt it will be Android. However, if any other OS that’s capable to compete with Android in terms of features and apps, then it would be iOS. Just like Android, iOS is also used by millions of users.
Although not much, iOS also offers users plenty of customization options. The app ecosystem of iOS is also rock solid and you will find apps for every different purpose. If you have been using iPhone for a while, then you might have come across an error message “Item you requested is currently not available in your country” on the app store.
So, what does that error really means? It means that the web content is available only in selected countries or regions and you can’t download that app. Let’s admit, there are times when we want to download trending apps for the iPhone, but we later find it that the app was Geographically restricted.
These little things often disappoint iPhone users. So, in this article, we have decided to share a working method that would help you install Georestricted apps on iOS devices like the iPhone. The method which we are going to share below is quite straightforward. So, let’s know how to install Georestricted apps on iPhone.
How to Install Georestricted Apps on iPhone
The method is quite simple and easy and you just need to follow some of the simple steps to proceed with this.
Step 1. First of all, head to the Settings > iTunes & App Store > (Your Apple ID). Under that page, tap on the ‘View Apple ID’ Option.
Step 2. On the next page, log in with your Apple ID and Password.
Step 3. On the next page, tap on the ‘Country/Region’ and on the next screen tap on ‘Change Country or Region’ option.
Step 4. Now you will be presented with a list of countries. Select the one as per your wish. Once done, tap on the ‘Agree’ button.
Step 5. Now you will be asked to fill in the billing address. You can fill any address of the selected country. Once done, click on the ‘Next’ button.
That’s it, you are done! Now your Apple store region has been changed. Now visit the app page and download the app. You can now download the geo-restricted app without any error.
So the above discussion is all about Install Georestricted Apps on iPhone. Use the guide and easily get your favorite apps that are not allowed to be downloaded in your country. Just use the trick and easily get the app. I hope you like this, keep on sharing with others too. Leave a comment below if you have any related queries with this.


{% extends "layout.html" %} {% block body %}
Enter the URL of a web-accessible image, and it will be served from an IIIF-compliant image server to either Mirador or Universal Viewer.
IIIF (the International Image Interoperability Framework) unlocks image content by providing a standard API for providers to expose images to visual tools for exploration, manipulation, annotation, combination, and more. We created this site to allow you to get a taste of what these tools (and IIIF) can do for you: enter any URL to a web-accessible image, and it will be served from an IIIF-compliant image server to either Mirador or Universal Viewer. We hope this will inspire you to consider adopting IIIF, or at least learn more about its capabilities.
{% endblock %} {% block aside %} {% endblock %}