class: center, middle # IEMS 5780
Building and Deploying Scalable
Machine Learning Services ### Lecture 7 - Network Programming #### Albert Au Yeung
25th October, 2019 --- class: center, middle # Computer Networking --- # Data Communication - Exchange of data between two devices using some form of transmission medium - A simplified communication model:
- When performing communication, we need **protocols**: rules that govern how data is transmitted in this system --- # Protocols ### What are protocols? - A set of **rules** that govern how communication parties interact with each other - An **agreement** between the communicating entities - Two devices need to agree on common protocols when they communicate - Things that a protocol should define: - The format of the **addressing scheme** - How do we specify the **start** and **end** of a data stream? - How do we **handle errors** or data loss? - How to handle problems in data transfer? - ... --- # Protocols ### An example: TCP's three way handshake
Reference: [https://hpbn.co/building-blocks-of-tcp/](https://hpbn.co/building-blocks-of-tcp/) --- # Layered Architecture - Computer networks usually adopt a **layered architecture** - Each layer has its own functions, and only interact directly with the one **above** and **below** it - Protocols are used to allow entity (e.g. a program) in one host to communicate with another entity in another host on the **same** layer ### Question - Why do we want to have a *layered* architecture? - Consider the example in the next slide --- # Layered Architecture - Imagine Beethoven is writing a letter to Einstein…
--- # Layered Architecture ### The ISO OSI (Open System Interconnection) 7-Layer Model - A **theoretical** model of how a computer network should work - It organises different functions of a network into **seven** different layers - It specifies the interfaces for communication between different layers and different endpoints ### Note: - It is a theoretical model - It is not a program or software - Practical networks may be implemented in a different way --- # OSI Model
--- class: split # OSI Model .column-left[
Source: [http://nhprice.com/what-is-ios-model-the-overall-explanation-of-ios-7-layers.html](http://nhprice.com/what-is-ios-model-the-overall-explanation-of-ios-7-layers.html) ] .column-right[ - Describes **what** need to be done to enable data to be sent from one device to another device - Each layer is a collection of similar functions - Each layer serves the layer **above** it - Each layer get services from the one **below** it ] --- # Layered Architecture ### What are the benefits of having a layered architecture? - It allows simplier design and implementation - Each layer focus on different functions - On each layer, protocols can be designed independently - Different applications may choose to implement data communication on different layers depending on their requirements --- # Layered Architecture ### The TCP/IP Protocol Suite
--- # TCP/IP
--- # Client/Server Model - A process, called a **client**, requests services from a process on another host, called a **server** - The following must be defined - Local host (Source IP address) - Local process (Source port number) - Remote host (Destination IP address) - Remote process (Destination port number) - In client-server model, if we regard the client as the local host, then the server is the remote host, and vice versa --- # TCP/IP ### There are three transport layer protocols defined in the TCP/IP Protocol Suite - **
User Datagram Protocol (UDP)
** - **
Transmission Control Protocol (TCP)
** - Stream Control Transmission Protocol (SCTP) - New reliable and message-oriented protocol combines the best features of UDP and TCP - For streaming applications (e.g. video streaming) --- class: split # TCP - The Transport Layer .column-left[ ### Connectionless vs. Connection-oriented - **
Connectionless (UDP)
** - No pre-established connection - Packets are not numbered, can arrive out of sequence - No acknowledgement of packets received - **
Connection-oriented (TCP)
** - A connection is first established between the sender and the receiver - Has transport layer-level flow and error control - Reliable ] .column-right[
vs.
] --- # UDP - User Datagram Potocol ### Characteristics of UDP - UDP is **connectionless** and **unreliable** - Very simple using a minimum of overhead - **Faster** and more efficient for many lightweight or time-sensitive purposes - Suitable for processes sending small messages and does not care much about reliability - Used for multicast and broadcast - Common network applications that use UDP: - Domain Name System (DNS) - Trivial File Transfer Protocol (TFTP) --- # Datagram ### UDP packets, called user datagrams, have a fixed-size 8 bytes header, containing 4 fields 1. **Source port number** - The port number used by the process running on the source host (16-bit) 2. **Destination port number** - The port number used by the process running on the destination host (16-bit) 3. **Length** - 16-bit field that defines the total length of the user datagram, header plus data (actually duplicated with the length field in IP) - UDP length = IP length – IP header’s length 4. **Checksum** - A checksum for the user datagram - *Question: how about IP address?* --- # TCP - Transmission Control Protocol ### TCP: a stream-oriented protocol - Instead of independent datagrams, TCP delivers data as **a stream of bytes** - A large chunk of data is divided into **segments**, these segments are related to one another - TCP creates an environment in which the two processes seem to be connected by an **imaginary tunnel** --- # TCP ### Flow Control - The sending and the receiving processes may not write or read data at the same speed - TCP needs **buffers for storage**, **flow control**, and **error control** - One way to implement the buffer is to use a circular array of 1-byte locations - TCP buffer size is configurable (e.g. buffer size = 2 x bandwidth x delay) (Can be up to megabytes) - UDP does not have buffers and its queue length is relatively smaller --- class: split # TCP .column-left[ - TCP delivers data as **segments** - TCP adds a header to each segment (for control purpose) and delivers the segment to the underlying IP layer for transmission - The segments are encapsulated in IP datagrams and transmitted (The entire operation is transparent to the processes) ] .column-right[
] --- class: center, middle # Network Programming and Socket Programming --- # Network Programming - What do we do in network programming?
--- # Ports - Ports are **"endpoints of communications"** in a computer's OS - Ports allow different applications running on the same computer to share a single physical link to the network - Each application must bind to a **unique port** (identified by a number) in order to communicate with the network --- # Ports
--- # Ports - Port number is a **16-bit unsigned integer** (i.e. 0 to 65535) - Port numbers are divided into **3 different ranges** (Regulated by the [Internet Assigned Numbers Authority (IANA)](https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml)) - **Well Know Ports** - (0-1023) Registered for well-known applications - Example: 21 (FTP), 80 (HTTP), 443 (HTTPS), 465 (SMTPS) - **Registered Ports** - (1024-49151) - Registered for other applications - **Dynamic/Private Ports** - (49151-65535) - Can be used by private applications --- # Using netstat - You can use `netstat` to check the active network connections on your computer ```bash $ netstat -a Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 *:microsoft-ds *:* LISTEN tcp 0 0 localhost:8002 *:* LISTEN tcp 0 0 *:8005 *:* LISTEN tcp 0 0 *:25672 *:* LISTEN tcp 0 0 *:8010 *:* LISTEN tcp 0 0 localhost:mysql *:* LISTEN tcp 0 0 localhost:6379 *:* LISTEN tcp 0 0 *:netbios-ssn *:* LISTEN tcp 0 0 *:http *:* LISTEN tcp 0 0 *:epmd *:* LISTEN tcp 0 0 localhost:8500 *:* LISTEN tcp 0 0 *:ssh *:* LISTEN ``` --- # The Client-Server Model - Many network applications follow the client-server model - In such a model, servers are continuously running to wait for the request from clients
--- # The Client-Server Model ### Note: - **“Client”** and **“server”** here refer to the role of the program at some instance - One application can be running both a client and a server at the same time - A mobile app can be a server, if it is serving data to another mobile app ### Questions: - Can you think of examples of **servers** and **clients**? - What would a server program do? What would a client program do? --- # A Server Program ### What does a server program do?
--- # A Client Program ### What does a client program do?
--- class: center, middle # Socket Programming in Python --- # Sockets - Used to identify a program (process) on a computer (host) - A socket is defined by two numbers: 1. **IP Address** (identifies the host) 2. **Port number** (identifies the process) - The commonly used implementation is called [Berkeley Sockets](https://en.wikipedia.org/wiki/Berkeley_sockets) or [POSIX sockets](https://en.wikipedia.org/wiki/POSIX) - Whenever a process on a host needs to talk to another process on another host, sockets must be created - **Sockets in Python** is an object-based interface to the low-level operating system calls that are normally used to accomplish networking tasks on POSIX-compliant operating systems - It exposes the normal **POSIX calls** for raw UDP and TCP connections --- # Server ### The Steps to create a server program 1. **Create** a socket object in your program 2. **Bind** the socket object to a socket in the computer 3. **Listen** for incoming connection from clients 4. **Loop**: 1. **Accept** connection from a client 2. **Do** some stuff 3. **Send** response to client (if necessary) 4. **Close** the connection --- # Using Sockets in Python ```python import socket # create an INET socket server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # bind the socket to the host and a port server_socket.bind((socket.gethostname(), 50001)) # Listen for incoming connections from clients server_socket.listen(10) ``` - **socket** is a module in the standard library that provides access to the socket interface [https://docs.python.org/3/library/socket.html](https://docs.python.org/3/library/socket.html) - `socket.socket()` returns a new socket - `socket.AF_INET` specifies that the socket will use the IPv4 family of address - The second argument can either be `socket.SOCK_STREAM` (TCP) or `socket.SOCK_DGRAM` (UDP) --- # Using Sockets in Python ```python import socket # create an INET socket server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # bind the socket to the host and a port server_socket.bind((socket.gethostname(), 50001)) # Listen for incoming connections from clients server_socket.listen(10) ``` - `server_socket` is an instance (object) of the socket class - `socket.gethostname()` will return the name of the machine (try it yourself), this tells the socket to bind to interface that is visible to the public - Instead, you can provide `"localhost"`, `"<IP_ADDRESS>"`, or even an empty string `""` - `50001` is the port number that you want the socket to bind to --- # Using Sockets in Python ```python import socket # create an INET socket server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # bind the socket to the host and a port server_socket.bind((socket.gethostname(), 50001)) # Listen for incoming connections from clients server_socket.listen(10) ``` - **NOTE**: `bind()` accepts only one argument, which is a tuple `(host_name, port_number)` - Ask the socket to listen for incoming connection by using `listen()` - `listen(10)` means that there can be at most 10 clients waiting in the queue (10 is the size of the **backlog**) --- # A TCP Server ### Remember the steps? 1.
**Create** a socket object in your program
2.
**Bind** the socket object to a socket in the computer
3.
**Listen** for incoming connection from clients
4. **Loop**: 1. **Accept** connection from a client 2. **Do** some stuff 3. **Send** response to client (if necessary) 4. **Close** the connection --- # A TCP Server ```python # ... # A indefinite loop while True: # accept connections from outside (client_socket, address) = server_socket.accept() # Read data from client and send it back data = client_socket.recv(1024) client_socket.sendall(data) # Close the socket client_socket.close() ``` - `accept()` is a **blocking** call (wait until a client tries to connect) - `recv(1024)` attempts to read 1,024 bytes from the client (also blocking) - `sendall(data)` attempts to send all bytes of `data` to the client --- # Sending and Receiving Data ### Behavour of `send()` - We used `sendall()` in the above example to send data, which waits until ALL data has been sent out - `send()` is a more generic function, which may or may not send out all data at once ```python >>> len(data) 50000 >>> client_socket.send(data) 34582 ... ``` - NOTE: TCP is a streaming protocol (remember `SOCKET.SOCK_STREAM`?), it does not have the concept of messages or records --- class: split # A TCP Server .column-left[ ### Notes: - During `accept()`, a **new** socket is created and used solely for communication with that particular client - Even though the client may send all data at the same time, `recv()` may not receive the data altogether - `recv()` is a blocking function here - It will not return until some data is received in the buffer - If it returns an empty string, it means that the client has **disconnected** (the connection is broken) ] .column-right[
] --- # A TCP Client ```python import socket # create an INET TCP socket soc = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # connect to the server (change localhost to an IP address if necessary) soc.connect(("localhost", 50001)) # Send a message to the server soc.send("Hello Server!".encode("utf-8")) # Receive data from the server data = soc.recv(1024) print(data.decode("utf-8")) # Always close the socket after use soc.close() ``` --- # Sending and Receiving Data ### Behaviour of `recv()` - It **blocks** until some data has arrived, and then it returns the data - Even though the client sends everything in one go, the data might NOT be received at the server in all in one block (due to the nature of underlying IP network) - If it returns with an empty string, it means that the connection has **dropped** ### Question - How do you know when to **stop** receiving? --- # A Protocol on Top of TCP ### Design your own protocol to exchange data between server and client 1. **Fixed length message** - A message is always of the same length (e.g. 1024 characters) - Call `recv` until you get the whole message 2. **Delimit your message** - Add a special string (e.g. `"##THE END##"`) at the end of a message - Call `recv` until you see that special string 3. **Indicate message length in a header** - Include a fixed length header at the beginning of the message - Call `recv` to get the header to determine message lenght, and then call `recv` until message is received --- # An Example of Fixed Lenght Message ```python # Fixed length of a message MSG_LENGTH = 2048 # Use a list to hold parts of the message received parts = [] bytes_received = 0 while bytes_received < MSG_LENGTH: part = soc.recv(min(MSG_LENGTH - bytes_received, 1024)) if part == b'': raise Exception("Connection is lost") parts.append(part) bytes_received += len(part) # Join the parts into a single message message = b"".join(parts) ``` --- # UDP Server ```python import socket # Note, the second parameter for socket() is socket.SOCK_DGRAM) server_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # Bind it to an IP address and a port server_socket.bind((socket.gethostname(), 50001)) while True: # data is the data sent from a client # address is the IP address of the client data, address = server_socket.recvfrom(1024) ``` - we no longer need to call `listen()` - we no longer need to call `accept()` (no connection will be established) --- # UDP Client ```python import socket client_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # Server address and port number, and message to be sent server_address = ("localhost", 50001) message = "I love socket programming in Python!" bytes_send = client_socket.sendto(message, server_address) # Close the socket socket.close() ``` --- class: center, middle # End of Lecture 7


Parses json and build the corresponding parsed JSON value.
Parsed JSON values are of the types num, String, bool, Null, Lists of parsed JSON values or Maps from String to parsed JSON values.
The optional revivier function, if provided, is called once for each object or list property parsed. The arguments are the property name (String) or list index (int), and the value is the parsed value. The return value of the revivier will be used as the value of that property instead the parsed value.
Throws FormatException if the input is not valid JSON text.
Serializes object into output stream.
Performs the same operations as stringify but outputs the resulting string to an existing StringBuffer instead of creating a new String.
If serialization fails by throwing, some data might have been added to output, but it won't contain valid JSON text.
Serializes object into a JSON string.
Directly serializable values are num, String, bool, and Null, as well as some List and Map values. For List, the elements must all be serializable. For Map, the keys must be String and the values must be serializable.
If a value is any other type is attempted serialized, a "toJson()" method is invoked on the object and the result, which must be a directly serializable value, is serialized instead of the original value.
If the object does not support this method, throws, or returns a value that is not directly serializable, a JsonUnsupportedObjectError exception is thrown. If the call throws (including the case where there is no nullary "toJson" method, the error is caught and stored in the JsonUnsupportedObjectError's cause
field.
If a List or Map contains a reference to itself, directly or through other lists or maps, it cannot be serialized and a JsonCyclicError is thrown.
Json Objects should not change during serialization. If an object is serialized more than once, stringify is allowed to cache the JSON text for it. I.e., if an object changes after it is first serialized, the new values may or may not be reflected in the result.
A JsonListener that builds data objects from the parser events.
This is a simple stack-based object builder. It keeps the most recently seen value in a variable, and uses it depending on the following event.
The current Map or List being built.
The most recently read property key.
Stack used to handle nested containers.
The current container is pushed on the stack when a new one is started. If the container is a Map, there is also a current key which is also stored on the stack.
The most recently read value.
Read out the final result of parsing a JSON string.
Pops the top container from the stack, including a key if applicable.
Pushes the currently active container (and key, if a Map).
Reports that an object could not be stringified due to cyclic references.
An object that references itself cannot be serialized by stringify. When the cycle is detected, a JsonCyclicError is thrown.
The first object that was detected as part of a cycle.
Returns a string representation of this object.
/ Implementation ///////////////////////////////////////////////////////////
Called on failure to parse source.
Parses source, or throws if it fails.
Parses a "false" literal starting at position.
sourceposition
must be "f".
Parses a "null" literal starting at position.
sourceposition
must be "n".
Parses a "true" literal starting at position.
sourceposition
must be "t".
Error thrown by JSON serialization if an object cannot be serialized.
The unsupportedObject field holds that object that failed to be serialized.
If an object isn't directly serializable, the serializer calls the 'toJson' method on the object. If that call fails, the error will be stored in the cause field. If the call returns an object that isn't directly serializable, the cause will be null.
The exception thrown by object's toJson
method, if any.
The object that could not be serialized.
Returns a string representation of this object.


Assembly
-
class
compas_assembly.datastructures.
Assembly
Bases:
compas_assembly.datastructures.Assembly
Methods
transform
(T)Transform an assembly by the given transformation matrix.
transformed
(T)Transform a copy of the assembly by the given transformation matrix.
Inherited Methods
__init__
()Initialize self.
add_block
(block[, key, attr_dict])Add a block to the assembly.
add_edge
(u, v[, attr_dict])Add an edge and specify its attributes.
add_node
([key, attr_dict])Add a node and specify its attributes (optional).
clear
()Clear all the network data.
connected_edges
(key)Return the edges connected to a node.
copy
()Make an independent copy of an assembly.
degree
(key)Return the number of neighbors of a node.
degree_in
(key)Return the numer of incoming neighbors of a node.
degree_out
(key)Return the number of outgoing neighbors of a node.
delete_edge
(u, v)Delete an edge from the network.
delete_node
(key)Delete a node from the graph.
edge_attribute
(key, name[, value])Get or set an attribute of an edge.
edge_attributes
(key[, names, values])Get or set multiple attributes of an edge.
edge_coordinates
(u, v[, axes])Return the coordinates of the start and end point of an edge.
edge_direction
(u, v)Return the direction vector of an edge.
edge_length
(u, v)Return the length of an edge.
edge_midpoint
(u, v)Return the location of the midpoint of an edge.
edge_point
(u, v[, t])Return the location of a point along an edge.
edge_vector
(u, v)Return the vector of an edge.
edges
([data])Iterate over the edges of the network.
edges_attribute
(name[, value, keys])Get or set an attribute of multiple edges.
edges_attributes
([names, values, keys])Get or set multiple attributes of multiple edges.
edges_where
(conditions[, data])Get edges for which a certain condition or set of conditions is true.
edges_where_predicate
(predicate[, data])Get edges for which a certain condition or set of conditions is true using a lambda function.
from_data
(data)Construct a graph from structured data.
from_edges
(edges)from_json
(filepath)Construct an assembly from the data contained in a JSON file.
from_lines
(lines[, precision])Construct a network from a set of lines represented by their start and end point coordinates.
from_networkx
(graph)from_nodes_and_edges
(nodes, edges)Construct a network from nodes and edges.
from_obj
(filepath[, precision])Construct a network from the data contained in an OBJ file.
Get the identifier of a random edge.
Get the identifiers of a set of random edges.
Get the identifier of a random node.
get_any_nodes
(n[, exclude_leaves])Get a list of identifiers of a random set of n nodes.
gkey_key
([precision])Returns a dictionary that maps geometric keys of a certain precision to the keys of the corresponding nodes.
has_edge
(u, v[, directed])Verify if the network contains a specific edge.
has_node
(key)Verify if a specific node is present in the network.
Returns a dictionary that maps the indices of a node list to keys in a node dictionary.
index_uv
()Returns a dictionary that maps edges in a list to the corresponding vertex key pairs.
is_leaf
(key)Verify if a node is a leaf.
is_node_connected
(key)Verify if a specific node is connected.
key_gkey
([precision])Returns a dictionary that maps node dictionary keys to the corresponding geometric key up to a certain precision.
Returns a dictionary that maps node dictionary keys to the corresponding index in a node list or array.
leaves
()Return all leaves of the network.
neighborhood
(key[, ring])Return the nodes in the neighborhood of a node.
neighbors
(key)Return the neighbors of a node.
neighbors_in
(key)Return the incoming neighbors of a node.
neighbors_out
(key)Return the outgoing neighbors of a node.
node_attribute
(key, name[, value])Get or set an attribute of a node.
node_attributes
(key[, names, values])Get or set multiple attributes of a node.
node_coordinates
(key[, axes])Return the coordinates of a node.
node_laplacian
(key)Return the vector from the node to the centroid of its 1-ring neighborhood.
Compute the centroid of the neighboring nodes.
nodes
([data])Iterate over the nodes of the network.
nodes_attribute
(name[, value, keys])Get or set an attribute of multiple nodes.
nodes_attributes
([names, values, keys])Get or set multiple attributes of multiple nodes.
nodes_where
(conditions[, data])Get nodes for which a certain condition or set of conditions is true.
nodes_where_predicate
(predicate[, data])Get nodes for which a certain condition or set of conditions is true using a lambda function.
Compute the number of edges of the network.
Compute the total number of interface nodes.
Compute the number of nodes of the network.
split_edge
(u, v[, t])Split and edge by inserting a node along its length.
subset
(keys)Create an assembly that is a subset of the urrent assembly.
summary
()Print a summary of the graph.
to_data
()Returns a dictionary of structured data representing the graph.
to_json
(filepath)Serialise the data dictionary representing an assembly to JSON and store in a file.
to_lines
()Return the lines of the network as pairs of start and end point coordinates.
Return the nodes and edges of a network.
to_obj
()Write the network to an OBJ file.
Return the coordinates of the network.
unset_edge_attribute
(key, name)Unset the attribute of an edge.
unset_node_attribute
(key, name)Unset the attribute of a node.
update_default_edge_attributes
([attr_dict])Update the default edge attributes.
update_default_node_attributes
([attr_dict])Update the default node attributes.
uv_index
()Returns a dictionary that maps edge keys (i.e.


Most Akan peoples live in Ghana, where they settled in successive waves of migration between the 11th and 18th centuries; others inhabit the eastern part of Côte d'Ivoire and parts of Togo.The Republic of Ghana is a country found in West Africa.
The Akan people of Ghana frequently name their children after the day of the week they were
born. Day names
are a cultural practice of the Akan people of Ghana and Ivory Coast. Everyday corresponds to a different
Akan male and female name.Notable individuals with akan names; Kwame Nkurumah,Koffi Annan,Adwoah Aboah
See More ...
The
names are;
This website will enable you to know your Akua name by providing your date of birth,month and year.Give it a try.


Building
Deploying
Monitoring
Applications
with Azure
https://dev.azure.com/knoxdevops/knoxdevops
Have you ever met someone very confident in their release process?
Someone so confident in their release process they deploy on Fridays?
Have you ever met someone with a manual release process?
This might be their attitude.
To reach sanity:
- Consistent builds
- Consistent deployments
- Monitor applications
Azure DevOps is the starting point.
https://dev.azure.com/knoxdevops/knoxdevops
I can store repos here, or GitHub, or Bitbucket, or wherever.
The language doesn't matter. I also have a PHP example here.
https://dev.azure.com/knoxdevops/php_sample
Consistent builds
Tip 1: Build on code commit!
Consistent builds
Tip 2: Keep your build definition in your repository.
pool:
vmImage: 'Ubuntu 16.04'
variables:
buildConfiguration: 'Release'
steps:
- script: |
dotnet build ./src/KnoxDevOps.API/ --configuration $(buildConfiguration)
dotnet test ./src/KnoxDevOps.Tests/ --configuration $(buildConfiguration) --logger "trx;LogFileName=testresults.trx"
dotnet test ./src/KnoxDevOps.Tests/ --configuration $(buildConfiguration) /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=$(System.DefaultWorkingDirectory)/TestResults/Coverage/
cd ./src/KnoxDevOps.Tests/
dotnet reportgenerator "-reports:$(System.DefaultWorkingDirectory)/TestResults/Coverage/coverage.cobertura.xml" "-targetdir:$(System.DefaultWorkingDirectory)/TestResults/Coverage/Reports" "-reportTypes:htmlInline" "-tag:$(Build.BuildNumber)"
cd ../../
dotnet publish ./src/KnoxDevOps.API/ --configuration $(buildConfiguration) --output $BUILD_ARTIFACTSTAGINGDIRECTORY
cd ./src/KnoxDevOps.API/
dotnet ef migrations script --output $BUILD_ARTIFACTSTAGINGDIRECTORY/database.sql --idempotent --context MeetupContext
- task: PublishTestResults@2
inputs:
testRunner: VSTest
testResultsFiles: '$(System.DefaultWorkingDirectory)/**/*.trx'
- task: PublishCodeCoverageResults@1
inputs:
codeCoverageTool: cobertura
summaryFileLocation: $(System.DefaultWorkingDirectory)/TestResults/Coverage/**/*.xml
reportDirectory: $(System.DefaultWorkingDirectory)/TestResults/Coverage/Reports
- task: PublishBuildArtifacts@1
Consistent builds
Consistent builds
Tip 3: Upload test results.
Consistent builds
Tip 4: Upload Code Coverage.
Consistent builds
- Build on code commit!
- Keep your build definition in your repository.
- Upload test results.
- Upload Code Coverage.
Azure DevOps Build Pipelines and your choice of source control.
Waiting on the build
Consistent deployment
AKA: I'm a dev, this isn't my problem.
Build Handoff to Operations
Existing Resources
Production, Testing, Dev, QA, Staging, or etc. environments would all look the same, have the
same resource types.
- App Service Plan - Allocated server resources
- App Service - Azure Web App
- Web App (staging slot) - Deployment slot sharing resources
- Application Insights - Logs, Analytics, Metrics
- Azure SQL Database
https://devops23.azurewebsites.net/api/values
Consistent deployments
Tip 1: Only release from one place
Consistent deployments
Tip 2: Define deployment stages
Staging slot
https://devops23-staging.azurewebsites.net/api/values
Staging slot
Swapping staging with production.
Tip 3: Automate every step of the deployment
Tip 3: Automate every step of the deployment
- Application APIs shouldn't break
- Database changes shouldn't be breaking changes
- A feature may be deployed through several deployments
Tip 4: Require QA/BA/PM approval.
Consistent deployments
- Only release from one place.
- Define deployment stages.
- Automate every step of the deployment.
- Require QA/BA/PM approval.
Azure DevOps Release Pipelines triggered by Azure DevOps Builds.
Monitor applications
Tip 1: Helpdesk tickets don't count as monitoring.
Support while everyone is here
The Failure
(the only piece of code, I swear)
Azure Application Insights
Gathers metrics about the attached applications.
Monitor Applications
A metric is a fact. A helpdesk ticket is a story.
Monitor Applications
Tip 2: Understand the rules you care about.
Monitor Applications
Tip 2: Understand the rules you care about.
- Do we care about a few errors if there's a retry policy?
- Are there defined response times agreements?
- Is every endpoint important?
Monitor Applications
Creating new rule for failed requests.
Monitor Applications
Alert someone when rule activated.
Monitor Applications
Cause failure.
Monitor Applications
Tip 3: Have tier 1 support on-call rotation.
Monitor Applications
- Helpdesk tickets aren't enough monitoring.
- Understand the rules you care about.
- Have tier 1 support on-call rotation.
- Pages should be urgent, important, actionable, and real. -Rob Ewaschuk
Azure Application Insights gathering information from Azure Web Apps.
Monitor Applications Bonus
Site Reliability
- SLI (Service Level Indicator)
- SLO (Service Level Objective)
- SLA (Service Level Agreement)
SLI (Service Level Indicator)
What can we measure and give a number for?
Average Website Response Time can be measured.
SLO (Service Level Objective)
What should the measurements be?
Average Website Response Time should be below 100ms.
SLA (Service Level Agreement)
What did we agree with our boss, that the measurements should be?
Who gets woken up at 2am to fix stuff?
Sanity:
- Consistent builds
- Consistent deployments
- Monitor applications
This is all Azure specific, or is it?... The same principles could apply to any project.
Daniel Oliver
Website: olivercoding.com
Twitter: @a_software_dev
Github: danieloliver
Slides: https://azure-app-talk.olivercoding.com/


THE LION KING A film review by Mike Huang Copyright 1994 Mike Huang
Disney Films Featuring: James Earl Jones, Whoopi Goldberg, Moira Kelly, Matthew Broderick, Jeremy Irons, Cheech Marin, and a host of others Music and Lyrics by Elton John and Tim Rice
Just when you thought that Disney could do no better; just when you thought that the pinnacle of an art form had been reached, Disney dared to push the envelope, and make another animated feature. To the pleasant satisfaction of many, and in classic Disney style, this film has outshined the sunrise over the African savanna.
Disney has had a grand tradition of animated film, which has spanned the greater course of this century. Starting with such films as SNOW WHITE, PINOCCHIO, and CINDERELLA, Walt Disney set the standard by which all other animated features have ever been measured.
This tradition wavered during the 1970's and early 80's, but made a sensational return with its trio of films, THE LITTLE MERMAID, BEAUTY AND THE BEAST, and ALADDIN. Each of these films went back to the tradition of a good story before excellent animation. The magic and charm of the art was brought back from a cellulose, near-death experience.
Throughout each of these films, not only did the Disney animation house draw deeper upon their artful imagery, they also drew upon stronger characters, and better vocal talents. To this end, they have drawn many a celebrity into the recording studio, including Robin Williams, James Earl Jones, and Jeremy Irons, the latter two both appearing in the Lion King.
As the grandeur and spectacular sights and sounds of ALADDIN thrilled audiences, and its timely, cutting-edge humor brought roarious laughter, and an occasional tear.
This film does one up from that, with spectacular vocal performances, brilliant sights and more than a plethora of the traditional Disney charm.
The artful expression of the characters was wonderful, ranging through the adorable Simba as a cub, to the evil Scar. Not to be outdone in expressing characters, Disney also went out of their way to create beautiful scenic backdrops, with a sense of depth panning across the wild African brush, often with hundreds of waterfowl flying across the land.
True to Disney style, the story is a tear-jerker, which is apparently geared more towards adults than children, but which most children will surely enjoy.
This grand drama of a lion cub finding himself and his true place in the call of the wild is a must-see, with some hilarious characterizations, and charm which brings back great drama and art true to the Disney tradition.
A definite must-see.
On a scale of 0 to 10 roars, this one gets 10.
-- Mike Huang<EMAIL_ADDRESS>.
The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


public class Progress
extends java.lang.Object
| Modifier and Type | Field and Description |
|---|---|
static int |
ALL
Flag for including all of the above in report
|
static int |
CHAR
Flag for including a spinner in report
|
static int |
CURRENT_TOTAL
Flag for including current/total in the reporting (example: 5/20)
|
static int |
ETA
Flag for including estimate time ahead (ETA) in report (example: ETA: 5
minutes)
|
static int |
PERCENTAGE
Flag for including percentage in the reporting (example: (25%))
|
| Constructor and Description |
|---|
Progress(int total)
Initializes a new instant of the Progress object.
|
| Modifier and Type | Method and Description |
|---|---|
double |
getEta()
Gets the estimated time remaining based on the time we used so fare
|
long |
getNowNano()
Gets the nano time passed since start of progress.
|
java.lang.String |
getNowPretty(boolean includeSeconds)
Gets the time passed as an human readable string.
|
double |
getProgress()
Gets the progress as a fraction
|
java.lang.String |
getProgressString(int options)
Gets the progress string.
|
int |
getTotal()
Get the final number for this progress
|
void |
increment()
Increases the progress by 1, but never beyond the total progress
|
void |
increment(int n)
Increases the progress by n, but never beyond the total progress
|
boolean |
isDone()
Returns true if this progress is at last state.
|
void |
printProgressToConsole(int options)
Erases previously printed line and prints the progress string.
|
void |
start()
Mark now as the starting point in negotiation
|
public static int CURRENT_TOTAL
public static int PERCENTAGE
public static int ETA
public static int CHAR
public static int ALL
public Progress(int total)
total
- The total amount of progress that can be madepublic void increment()
public void increment(int n)
n
- the value to be addedpublic boolean isDone()
public void start()
public long getNowNano()
public java.lang.String getNowPretty(boolean includeSeconds)
includeSeconds
- if true, will print up to seconds. if false, will print up to
minutes.public int getTotal()
public double getProgress()
public java.lang.String getProgressString(int options)
Progress.ETA: Report estimated time ahead Progress.CURRENT_TOTAL: Report current/total Progress.PERCENTAGE: Report percentage done Progress.CHAR: Show spinner char Progress.ALL: All of the above
options
- See descriptionpublic void printProgressToConsole(int options)
Progress.ETA: Report estimated time ahead Progress.CURRENT_TOTAL: Report current/total Progress.PERCENTAGE: Report percentage done Progress.CHAR: Show spinner char Progress.ALL: All of the above
options
- See descriptionpublic double getEta()


I'm an allround developer used to work with multiple languages in different environments. You find my recent projects further down the line.
DigiD
In the DigiD project I was member of the reporting team. I developed queries and csv reports in Ruby on Rails.
All reports had unit tests in rspec and where deployed with Capistrano.
For real-time reporting I used rails queries feeding a munin frontend.
BelTel
A distributed contact center,
build around Genesys middleware. Backend is in Java, the telephony stack is build on Genesys middleware and HiPath telephone
systems. The frontend is build with Visual Basic 6.
My roles included frontend / backend tester, administrator for the D&T environment and finally
maintainer for the frontend application.


I'm a software engineer currently working at Instructure. I like to create niche but useful tools for myself and others. I also dabble in music production and game development.
Developing Canvas LTIs and web applications using Ruby on Rails, PostgreSQL, JavaScript, Docker, React, and Redux.
Assisted users with Canvas questions/issues via chat and phone. Developed pfollowup, a support tool that allows our agents to quickly send detailed followups via Salesforce.
I've made contributions to Tiled, a popular map editor. I added multiple property editing which has saved me tons of time in my Tiled workflow.
View ProjectAn app made with Python that automatically slices sprites given an image. This functionality is built into Unity now, so not much use for this in my personal projects. However I learned a lot with this project, and it was a lot of fun.
View ProjectTurns any image into a party gif. Can be customized to create any other type of colorful gif. Great for Slack emojis!
View ProjectBubble Pop Mania was an old flash game I made for Newgrounds. This is a remake made with Unity.
View Project


Networks of citations are a key tool for referencing, disseminating and evaluating research results. The task of characterising the functional role of citations in scientific literature is very difficult, not only for software agents but for humans, too. The main problem is that the mental models of different annotators hardly ever converge to a single shared opinion. The goal of this paper is to investigate how an existing reference model for classifying citations, namely CiTO (Citation Typing Ontology), is interpreted and used by annotators of scientific literature. We present some experiments capturing the cognitive processes behind users' decisions in annotating papers with CITO, and provide initial ideas to refine future releases of CiTO and to simulate readers' behaviour within CiTaLO, a tool for automatic classification of citations.
The interest in alternative ways for publishing scientific results is rapidly increasing, especially for the Semantic Web community that is producing a lot of scientific data as Linked datasets which can be browsed and reasoned on . On the other hand, most of the current scientific production is still disseminated by “traditional” papers, and citations remain the key tools to connect, explore and evaluate research works.
Citations are not all equal, as discussed by . The frequency a work is cited is a partial indicator of its relevance for a community. More effective results can be obtained by looking for the citation functions, i.e. “the author's reasons for citing a given paper” . Yet, it is extremely difficult to characterise the nature of a citation univocally.
A fairly successful classification model is CiTO (Citation Typing Ontology) , an OWL ontology for describing factual as well as rhetorical functions of citations in scientific articles and other scholarly works. CiTO defines forty-one properties that allow users to characterise precisely the semantics of a citation act. CiTO has been successfully used in large projects like CiteULike and Data.open.ac.uk and several tools have been developed to annotate citations with CiTO properties directly via browser or CMS plugins .
Despite (or possibly because of) the richness and variety of CiTO properties, most users actually employ a sub-set of these properties: a smaller set of properties is easier to memorise and handle, some properties are not perceived as precise in specific domains, some others are considered too similar to each others, and so on. For instance, Pensoft Publishers (to our knowledge, the first commercial user of CiTO) are going to enable authors to annotate their citations according to six CiTO properties only. The Link to Link Wordpress plugin supports about ten properties.
Clearly some work restructuring the ontology is in order. This paper in fact presents an experimental analysis on how the CiTO model is accepted, understood, and adopted by users. It discusses tests performed in parallel both on the full set of CiTO properties and on a specific subset, along with the feedback directly provided by the users. The goal is to validate and assess the usability of CiTO and to distill guidelines for a more effective use of the current ontology and for improvements to the future releases. Also, we want to study user behaviour in order to simulate it within CiTalO , a chain of tools for identifying automatically the nature of citations.
The most critical aspect identified by our tests is that opinions about the most appropriate CiTO properties are often misaligned. Apparently each reader relies on a different mental model, and the reader's model can be and often is different from the one of the authors of the CiTO annotation, which in turn can be and often is different from the one of the authors of the citation in the paper.
This paper is structured as follows: in we review previous works on classification of citations. In we present CiTO and relate it with users' mental models. In we introduce our experimental setting and findings. In and we discuss the lesson learnt and sketch out some possible developments of CiTO and CiTalO.
The analysis of networks of citations is gaining more and more attention. Copestake et al. present an infrastructure, called SciBorg, based on NLP techniques that allows one to automatically extract semantic characterisations of scientific texts. In particular, they developed a module for discourse and citation analysis based on the approach proposed by Teufel et al. called Argumentative Zoning (AZ). AZ provides a procedural mechanism to annotate sentences of an article according to one out of seven classes of a given annotation scheme (i.e. background, own, aim, textual, contrast, basis and other), thus interpreting the intended authors' motivation behind scientific content and citations.
Teufel et al. study the function of citations – that they define as “author's reason for citing a given paper” – and provide a categorisation of possible citation functions organised in twelve classes, in turn clustered in Negative, Neutral and Positive rhetorical functions. In addition, they performed some tests involving hundreds of articles in computational linguistics (stored as XML files), several human annotators and a machine learning approach for the automatic annotation of citation functions. The results were quite promising; however the agreement between human annotators (i.e. K = 0.72) is still higher than the one between the human annotators and the machine learning approach (i.e. K = 0.57).
Jorg analysed the ACL Anthology Networks and found one hundred fifty cue verbs, i.e. verbs usually used to carry important information about the nature of citations: based on, outperform, focus on, extend, etc. She maps cue verbs to classes of citation functions according to the classification provided by Moravcsik et al. and makes the bases to the development of a formal citation ontology. This works actually represent one of the sources of inspiration of CiTO (the Citation Typing Ontology) developed by Peroni et al. , which is an ontology that permits the motivations of an author when referring to another document to be captured and described by using Semantic Web technologies such as RDF and OWL.
Closely related to the annotation of citation functions, Athar proposes a sentiment-analysis approach to citations, so as to identify whether a particular act of citing was done with positive (e.g. praising a previous work on a certain topic) or negative intentions (e.g. criticising the results obtained through a particular method). Starting from empirical results Athar et al. expand the above study and show how the correct sentiment (in particular, a negative sentiment) of a particular citation usually does not emerge from the citation sentence – i.e. the sentence that contains the actual pointer to the bibliographic reference of the cited paper. Rather, it actually becomes evident in the last part of the considered context window .
Hou et al. use an alternative approach to understand the relevance (seen as a form of positive connotation/sentiment) of citations: the citation counting in a text. Paraphrasing the authors, the idea is that the more a paper is cited within a text, the more its scientific contribution is significative.
There are several reference models to characterise citations, as presented in the previous section. One of the most used within the Semantic Web domain is CiTO . The ontology basically defines a property cites (and its inverse isCitedBy) and 41 sub-properties (each of which has its own inverse) that describe the semantics of a citation act.
The richness of properties is a key feature of CiTO. To the best of our knowledge, there is no other OWL ontology that provides a set of properties for annotating citation types as rich as CiTO. This aspect has contributed to the adoption of the ontology by the Semantic Publishing community, which is currently exploiting CiTO in projects like CiteULike, Data.open.ac.uk, and the Open Citation Corpus .
On the other hand, the richness of CiTO is perceived as a hindrance by some annotators. We studied the actual adoption of the model within these projects and discovered that most tools actually employ a sub-set of the CiTO properties (CiTO-Ps on the rest of the text). For instance, Pensoft Publishers are going to enable authors to annotate their citations according to only six CiTO-Ps – i.e., citesAsDataSource, related, critiques, supports, reviews and discusses – while the Link to Link Wordpress plugin allows users to specify the generic function cites and some of its sub-properties: citesAsSourceDocument, confirms, extends, obtainsBackgroundFrom, reviews, supports, usesDataFrom, usesMethodIn, and disagreesWith.
We believe that one of the reasons for this fragmented adoption is that CiTO was developed with a top-down approach: the authors of the ontology, supported by a group of experts and end-users, and with the help of previous works on this topic, studied collections of scientific papers and citation patterns and came up with a set of properties that was incrementally refined.
The goal of this work is to assess and validate the CiTO-Ps from a bottom-up perspective. This approach is complementary to the current CiTO development process and allows us to study how CiTO-Ps are actually perceived by users in the task of annotating citations.
One of the most relevant issues we found is that multiple views coexist and often conflict when performing an annotation task with CiTO. We can see three steps in this process: (i) the interpretation of the text so as to guess the citation function as it was originally conceived by the author, (ii) the understanding of the CiTO-Ps and (iii) the creation of a mapping between the supposed function of a citation and the most appropriate CiTO property.
For each step, each annotator creates her/his own mental model. Mental models were introduced in 1943 by Craik as small-scale models of reality that the humans' mind uses to anticipate events. For most cognitive scientists today, a mental model is an internal scale-model representation of an external reality. It is built on-the-fly, from knowledge of prior experience, schema segments, perception, and problem-solving strategies. In Human-Computer Interaction (HCI) mental models are detected for improving the usability of a system. It is commonly accepted that humans interact with systems based on a set of beliefs about how a system works.
This also applies to CiTO processes: users annotate a citation based on a set of beliefs that they obtain by only interpreting the citation's context and finding an appropriate property in CiTO according to their interpretation of the ontology. Our goal is to study these beliefs and the mental models built by the users. The work is based on the hypothesis that usability is tied strongly to the extent to which these models match and predicts the action of a system. The overall objective is to reduce the gap between the System Model, the mental model constructed by the ontology engineer while modelling CiTO-Ps, and the User Model, the mental model constructed by a user for understanding how to use CiTO-Ps.
In order to assess how CiTO is used to annotate scholarly articles, we compared the classifications performed by humans on a set of citations. The tests involved twenty users with different background and skills. We meant to collect a set of quantitative indicators to answer the following numbered research questions (RQn) on CiTO:
Which properties have been used by users during the test?
Which were the most used properties?
What was the global inter-rater agreement of the users?
Did the number of available choices bias the global inter-rater agreement?
Which properties showed an acceptable positive agreement in users?
Could properties be clustered according to their similarity in users' annotations?
What was the perceived usability of the ontology?
Which were the features of CiTO-Ps that users perceived as most useful or problematic?
Additionally, after the completion of the annotation task, we asked each user to fill two questionnaires: a multiple-choice questionnaire aimed at measuring the System Usability Scale (SUS) , and a second questionnaire with free-text answers to capture the users' satisfaction in using CiTO-Ps for annotating citations.
In order to simplify the task of annotating citations, we prepared and normalized how the citations were presented to the users. Identifying the boundaries of a citation, or better which boundaries are needed to capture the nature of that citation, is not a trivial task: the citation sentence, i.e. the sentence containing directly the citation, often is not enough. As confirmed by Athar et al. , the actual intended sentiment and motivation of a citation might be explicated in other sentences close to the citation and can refer implicitly (i.e. by means of implicit citations ) to the cited work (through authors' names, project's name, pronouns, etc.). This issue is known as the identification of context window . Even if there are multiple techniques for automatic extraction of the context window, taking care also of implicit citations, in this experiment we identified them manually: we read the text and tried to understand which sequence of sentences around a particular citation conveyed its citation function at the best.
The test bed includes some scientific papers encoded in XML DocBook chosen among the
seventh volume of the proceedings Balisage Conference Series . We automatically extracted citation sentences, through an XSLT
transform (available at http://www.essepuntato.it/2013/citalo/xslt
). We took into account only those
papers for which the XSLT transform retrieved at least one citation (i.e. 18 papers
written by different authors). The total number of citations retrieved was 377, for a mean
of 20.94 citations per paper.
We then filtered all the citation sentences that contain verbs (extends, discusses, etc.) and/or other grammatical structures (uses method in, uses data from, etc.) that carry explicitly a citation function. We considered that rule as a strict guideline as also suggested by Teufel et al. . We obtained 105 citations out of 377, obtaining at least one citation for each of the 18 paper used (a mean of 5.83 citations per paper). These citations are very heterogeneous and provide us a significative sample for out tests. Finally, we manually expanded each citation sentence (i.e. the sentence containing the reference to a bibliographic entity) selecting a context window that we think is useful to classify that citation, as explained above.
The test was carried on by two groups, each one composed by ten users, by means of a Web interface . The first group could select properties out of the full list of 41 CiTO-Ps (T41 from now on). Instead, the second one performed the same task by only using 10 CiTO-Ps (T10). This reduced set of properties comes from a preliminary experiment we undertook in that showed that only these properties had a moderate inter-rater agreement (Fleiss' k > 0.33). The goal in fact is to answer RQ4: does the number of properties affect the inter-rater agreement?
Both groups were composed mainly by computer scientists (the main area of the Balisage Conference), none an expert user of CiTO. Each user read each citation sentence separately, with its full context window, and had to select one CiTO-Ps for that sentence. Users could also revise their choices and perform the experiments off-line. There was no time constraint and users could freely access the CiTO documentation.
All the data collected were stored in RDF and we used R to load the data and elaborate the results .
The experiments confirmed some of our hypotheses and highlighted some unexpected issues too. The first point to notice is that our users have selected 37 different CiTO properties over 41 in T41, with an average of 21.7 properties per user, while they have selected all the 10 properties in T10 (the mean by user is 10) (RQ1). Moreover, in T41 a few of these properties have been used many times, while most of them have been selected in a small number of cases, as shown in – this table answers to RQ2.
| CiTO property in T41 | # in T41 | CiTO property in T10 | # in T10 |
|---|---|---|---|
|
citesForInformation |
151 |
citesForInformation |
190 |
|
citesAsRelated |
122 |
obtainsBackgroundFrom |
152 |
|
citesAsAuthority |
85 |
citesAsRelated |
137 |
|
citesAsRecommendedReading |
72 |
citesAsDataSource |
126 |
|
usesMethodIn, citesAsSourceDocument, citesAsPotentialSolution, credits, citesAsDataSource, citesAsEvidence, |
< 72 |
citesAsRecommendedReading |
116 |
|
describes, obtainsSupportFrom, extends, obtainsBackgroundFrom, usesDataFrom, agreesWith, critiques |
< 40 |
credits |
86 |
|
discusses, usesConclusionsFrom, confirms, containsAssertionFrom, includesQuotationFrom, supports, citesAsMetadataDocument, reviews, documents |
< 18 |
citesAsPotentialSolution, usesMethodIn |
80 |
|
updates, disputes, compiles, corrects, qualifies, disagreesWith, includesExcerptFrom, refutes, speculatesOn, derides, retracts |
< 6 |
critiques, includesQuotationFrom |
< 80 |
In T41, there were 4 properties not selected by any user: parodies, plagiarizes, repliesTo and ridicules. This is not surprising considering the meaning of these properties.
These data show that there is a great variety in the choices of humans. In fact, only 18 citations in T41 and 24 citations in T10 (out of 105) have been classified with exactly the same CiTO property by at least 6 users. These results are summarised in , together with the list of selected properties for both T41 and T10. We indicate how many citations of the dataset users agreed on, and the number of properties selected by the users.
| Test | # Citations in the dataset | CiTO properties |
|---|---|---|
|
T41 |
18 |
citesForInformation (28), citesAsRelated (26), citesAsPotentialSolution (21), citesAsDataSource (19), citesAsRecommendedReading (17), citesAsAuthority (16), usesDataFrom (7), agreesWith (6), citesAsSourceDocument (6), usesMethodIn (6), confirms (5), credits (5), obtainsSupportFrom (5), supports (4), citesAsEvidence (3), compiles (2), describes (2), obtainsBackgroundFrom (2) |
|
T10 |
24 |
citesAsDataSource (56), citesAsRecommendedReading (43), citesForInformation (35), obtainsBackgroundFrom (30), citesAsRelated (26), includesQuotationFrom (16), critiques (15), citesAsPotentialSolution (11), credits (5), usesMethodIn (3) |
Considering all the 105 citations, the agreement among humans was very poor in both T41 and T10. In fact we measured the Fleiss' k (that assesses the reliability of agreement between a fixed number of raters classifying item) for the 10 raters over all 105 subjects and obtained k = 0.13 in T41 and k = 0.15 in T10, meaning that there exists a positive agreement between users but it is very low – this answers to RQ3. In addition, the use of a larger number of CiTO-Ps (in T41 compared to T10) does not seem to affect the inter-rater agreement among users (RQ4).
Another very interesting finding is that users eventually agree only on one property per test. Even considering the whole dataset whose k value was very low, we found a moderate positive local agreement (i.e. 0.33 < k < 0.66) on citesAsPotentialSolution in T41 and on includesQuotationFrom in T10 – this answers to RQ5.
In order to identify other properties that showed a partial positive local agreement among users, we filtered only the 18 (in T41) and 24 (in T10) citations on which at least 6 users used the same property, as mentioned earlier in . The k value on that subset showed a moderate positive agreement: k = 0.39 in T41 and k = 0.43 in T10. In addition, we had k > 0.5 for 5 CiTO-Ps in T41 – i.e., agreesWith (k = 0.54), citesAsDataSource (k = 0.52), citesAsPotentialSolution (k = 0.66), citesAsRecommendedReading (k = 0.6), usesMethodIn (k = 0.54) – and for 4 CiTO-Ps in T10 – citesAsDataSource (k = 0.63), citesAsPotentialSolution (k = 0.71), citesAsRecommendedReading (k = 0.52, includesQuotationFrom (k = 0.69).
Given this heterogeneity, we tried to identify clusters of properties and to understand whether the testers selected the same properties together. The goal is to found which properties have similar meaning according to users' annotation. To do so, we applied the Chinese Whispers clustering algorithm to the graphs of collocates obtained by looking at the annotations provided by each user for each citation in T41. The graphs were built as follows. For each citation, we considered all the combinations of pairs of different CiTO-Ps as annotated by users, considering repetitions in the graph Gr and without repetitions in the graph Gn – it means that, for instance, having three users' annotations of a citation – e.g., extends, extends, and updates – we generated two pairs in G1 – i.e., (extends,updates) and (extends,updates) – and one pair in G2 – i.e., (extends,updates). In fact we were interested in highlighting recurrent collocates when CiTO-Ps were used at local level (Gr) or at a global level (Gn). Then, we created an edge linking two nodes (i.e., two different CiTO-Ps) for each collocate and we weighted it according to how many times that collocate is repeated in the dataset. We run the Chinese Whisper algorithm for 20 iterations on each graph and we observed that:
in Gr, considering only those arcs having weight at least 3, the algorithm returned a small cluster composed by the CiTO-Ps disputes, critics, derides and refutes;
in Gn, considering only those arcs having weight at least 5, the algorithm returned another small cluster composed by the CiTO-Ps credits, confirms and obtainsSupportFrom.
The algorithm results seem to indicate that there exist some sort of relations (e.g., taxonomical, equivalence, etc.) among the CiTO-Ps of each cluster and, to some extent, they can be used in an interchangeable way when annotating the citations – this answer to RQ6.
At the end of the test, both groups of testers were asked to answer to a SUS questionnaire including some free-text answering fields – in order to get some feedback on CiTO. The usability score for CiTO-Ps was computed using the System Usability Scale (SUS) , a well-known metrics used for the perception of the usability of a system. It has the advantage of being technology independent and it is reliable even with a very small sample size . In addition to the main SUS scale, we also were interested in examining the sub-scales of pure Usability and pure Learnability of the system, as proposed recently by Lewis and Sauro . As shown in , the mean SUS score for CiTO-Ps in T41 was 53.5 while in T10 was 62.5 (in a 0 to 100 range). The mean values for the SUS sub-scales Usability and Learnability were, respectively, 50.94 and 63.7 in T41 and 60.94 and 68.7 in T10. However, the only difference approaching the statistical significance (i.e., 0.05 < p < 0.1) was found between the Usability measures (i.e., p = 0.06), suggesting that the perceived usability of CiTO-Ps in T10 is better than that of CiTO-Ps in T41 – this answer to RQ7.
| CiTO-Ps | SUS mean | Usability mean | Learnability mean |
|---|---|---|---|
|
CiTO-Ps in T41 |
53.5 (s.d. 14.5) |
50.94 (s.d. 12.42) |
63.7 (s.d. 25.99) |
|
CiTO-Ps in T10 |
62.5 (s.d. 11.79) |
60.94 (s.d. 10.13) |
68.7 (s.d. 25.85) |
The final text questionnaire contained a few questions, two asking for positive aspects, and two for negative aspects of CITO, and orthogonally two asking for qualifications (i.e., adjectives), and two for features (i.e., substantives) of the tool:
How effectively did CiTO properties support you in answering to the previous tasks?
What were the most useful features (labels, descriptions, examples, etc.) of CiTO properties to help you realise your tasks?
What were the main weaknesses that CiTO properties exhibited in supporting your tasks?
Can you think of any additional features that would have helped you to accomplish your tasks?
A fifth question was added to propose a discussion about the sheer size of the list of CITO properties:
Considering the experiment you have just completed, do you think that the number of CiTO properties was:
All 20 (10 in T41 and 10 in T10) subjects produced relevant content for the questions.
In order to obtain some meaningful results, we subjected the text answers to a grounded theory analysis. Grounded theory is a method often used in Social Science to extract relevant concepts from unstructured corpora of natural language resources (e.g., texts, interviews, or questionnaires). In opposition to traditional methods aiming at fitting (and sometimes forcing) the content of the resources into a prefabricated model, grounded theory aims at having the underlying model emerge “naturally” from the systematic collection, rephrasing, reorganisation and interpretations of the actual sentences and terms of the resources. We thus believe it is a reasonable tool to examine our questionnaires in order to let relevant concepts emerge from the analysis.
We proceeded first with open coding, with the purpose of extracting actual relevant sentences – called codes – from the texts, and subsequently performed the so-called axial coding, which is the rephrasing of the original codes so as to have semantic connections emerge from them and generate concepts. We finally analysed the respective frequency of each emerged concept (defined as the number of codes which contributed to the concept's existence) so as to consider the most important issues arising from the answers.
Coding was performed separately in T41 and T10, but a later effort to homogenise the concepts drawn from the two sets was performed, so that results from the two tests could be compared. shows the results of those codes that were mentioned at least twice.
Some interesting suggestions came up from these data – this answers to RQ8.
Need to improve labels. The first question basically asked to identify the best between property label, property description and example, in conveying the best use of the property. Users massively preferred examples, with descriptions in the middle and labels last. This clearly indicates that CiTO labels should be improved to capture the nuances in the semantics of the CITO-Ps.
Need for a structure in the properties. It was evident the perception that many properties, for good or worse, overlapped semantically, often forcing a choice between similar properties rather than offering a clear and natural candidate. The issue of the hierarchy was particularly felt by the T41 testers, working with a flat list of 41 items: nine people (i.e., all but one) complained about this fact and asked for structured guidance. Another difference between the two tests (probably obvious in hindsight), is that testers in T41 had issues with the sheer number of properties (six individuals discussed about this) and with a feeling of imbalance in how they addressed the semantic scope of the citations. This suggests that the structuring of the properties loudly asked by the testers should also consider an adequate balancing of the properties, without exaggerating with negative ones, with sentiment-loaded ones, or with relationships.
Need for uniformity and bi-directional properties. A final mention should be given, in our view, to specific ideas and suggestions that, although provided by one tester each, are nonetheless interesting and worth further exploration: making all properties bi-directional (so that we could have both “gives support to” and “receives support from”), providing a decision tree for properties (so as to simplify the task of choosing the right one), or helping with the use of statistics (e.g., 30% of citations are request for information, 6% are credits, etc.) which also could provide guidance, if not for the best candidate property, at least for uniformity in choices between different annotators.
The starting point of this paper was that the characterisation of citations is an extremely difficult task also for humans. We presented some experiments to investigate which are the main difficulties behind this characterisation and, in particular, how the users understand and adopt CiTO based on the mental models they construct for addressing the annotations task. Our analysis – both experimental data and users' direct feedback – gave us some indications to improve CiTO and to increase its effectiveness, that we would summarise as follows.
Reduce the number of less-used properties. One of our findings was that some of the CiTO-Ps in T41 were used only few times or not used at all. This result can depend on a variety of factors. First, the authors of the articles in our dataset, which are researchers on markup languages, use a quite specific jargon so the context windows resulted not easy to interpret with respect to citations. Second, the positive or negative connotation of the properties was difficult to appreciate. For instance, the fact that the properties carrying negative judgements (disagreesWith, disputes, parodies, plagiarizes, refutes, repliesTo, ridicules, etc.) are less frequent than neutral and positive ones supports the findings of Teufel et al. on this topic. Notice also that, as highlighted by , a criticism can be postponed far from the sentence containing the citation and can be prefaced with positive feedback; where ever the criticism occur, it can be “toned down, disguised, or redirected away from important people” .
Identify the most-used neutral properties. Although we think the intended audience of the research articles one choose for such an experiment may bias the use of some properties, we also believe that some properties are actually shared among different scholarly domains. The property citesForInformation and citesAsRelated are a clear example. As expected, they were the most used properties, being the most neutral ones of CiTO. This is in line with the findings of Teufel et al. , on the analysis of citations within Linguistics scholarly literature. In that paper, the neutral category Neut was used for the majority of annotations by humans. Although their large adoption, citesForInformation and citesAsRelated had a very low positive local agreement (k = 0.07 and k = 0.2 respectively). This is not surprising since the properties were used many times, often as neutral classification on citations that were classified in a more precise way by other users. Note that one particular user in T41 identified credits as the most used (and, thus, neutral) property, which is also confirmed by running the Chinese Whispers algorithm on Gr, considering only those arcs having weight at least 5, that showed how credits formed a cluster of one element only and was linked to other more specific properties such as citesAsAuthority, citesAsDataSource, discusses, etc.
Investigate motivations for low inter-rater agreement. The main reason for running the two test T41 and T10 was to investigate whether the high number (i.e., 41) of CiTO-Ps could be a justification for obtaining a so low positive agreement in total (i.e., k = 0.13), due to the cognitive effort users spent to choose among such huge set of properties for annotating citations. Intuitively, a reduced number of properties should reduce the cognitive effort required by the users in building their mental models for understanding CiTO-Ps for annotating a given set of citations. Although the comparison of the SUS values obtained for T41 (which involved 41 CiTO-Ps) and T10 (where we asked to use only 10 CiTO-Ps) was in favour of the latter, i.e., the users perceived the small set of properties as more usable of the full set for annotating citations, the inter-rater agreements obtained in T41 (i.e., k = 0.13) and T10 (i.e., k = 0.15) show how the number of available CiTO-Ps did not actually impact too much. We have also tried to consider the results obtained in the two test if only expert users of citations (i.e., professors, academic researchers, postdoc and last year PhD students, representing half of the users in each test) were involved, and we noticed that the inter-rater agreements do not change at all for both tests. It seems that the number of CiTO-Ps and the kinds of users are not the main factors to take into account for that low agreement.
Define explicit relations between CiTO properties. since there is no hierarchical structure , each user followed its own mapping determined by the mental model built and ended up selecting very different values – probably because users' mental models differed largely between users. Our opinion is that a further investigation is needed on the structure of CiTO properties. To this end, the identification of clusters of properties, such as those introduced in the previous section, could be a possible way to follow, as well as the use of other approaches, i.e., pure statistical techniques for assessing collocates of CiTO-Ps (e.g., the use of chi-square test to analyse bigrams of collocates ) and empirical algorithms used for identifying relations (taxonomical, equivalence, meronomy, etc.) among keywords (e.g., Klink ). These approaches could be useful, for instance, to suggest ways to separate general properties form more specific ones or to build one or multiple hierarchies over the list of properties.
Add support for customised properties. One of the aspects of CiTO that users suggested to improve is its support for customisation. In some cases users could not find a property that perfectly fit their needs: they selected the one apparently closest to their mental model but they perceived this as a limitation of the model. To solve this issue, the latest release of CiTO – supported by our findings – includes an extension mechanism that allow users to use their own citation function. The syntax is briefly shown below:
@prefix cito: <http://purl.org/spar/cito/> . @prefix oa:
<http://www.w3.org/ns/oa#> . @prefix dcterms: <http://purl.org/dc/terms/> . [] a
oa:Annotation ; oa:hasBody [ dcterms:description "The cited paper initiated a whole field of
research" ] ; oa:hasTarget [ a cito:CitationAct ; cito:hasCitingEntity
<uri-citing-paper> ; cito:hasCitationEvent cito:cites ; cito:hasCitedEntity
<uri-cited-paper> ] .
The interesting aspect is that the overall organisation of the ontology (i.e., the TBox) does not change, while users are free to express precisely their characterisation capturing details and tones.
Extend examples, labels and explanations. The fact that some properties were misunderstood by the users – or the same property interpreted in different ways – is an indicator of the need for improvements in examples shown in our tests. One possible way of improving them will be to use the citations with the highest agreement and create additional samples from them.
These findings will also provide a basis for improving CiTalO , a tool for identifying automatically the nature of citations. In particular, we plan to in our ongoing work to investigate cognitive architecture in order to extend CiTalO to simulate users' behaviour determined by mental models.
We would like to thank all the people who participated to our (intensive and stressing) evaluation.
Peroni, S., Shotton, D. (2012). FaBiO and CiTO: ontologies for describing bibliographic resources and citations. In Journal of Web Semantics, 17 (December 2012): 33-43. DOI: 10.1016/j.websem.2012.08.001
Hou, W., Li, M., Niu, D. (2011). Counting citations in texts rather than reference lists to improve the accuracy of assessing scientific contribution. In BioEssays, 33 (10): 724-727. DOI: 10.1002/bies.201100067
Copestake, A., Corbett, P., Murray-Rust, P., Rupp, C. J., Siddharthan, A., Teufel, S., Waldron, B. (2006). An architecture for language processing for scientific text. In Proceedings of the UK e-Science All Hands Meeting 2006.
Athar, A., Teufel, S. (2012). Context-Enhanced Citation Sentiment Detection. In Proceedings of Human Language Technologies: Conference of the North American Chapter of the Association of Computational Linguistics 2012: 597-601.
Athar, A. (2011). Sentiment Analysis of Citations using Sentence Structure-Based Features. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: 81-87.
Teufel, S., Siddharthan, A., Tidhar, D. (2006). Automatic classification of citation function. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing: 103-110.
Teufel, S., Siddharthan, A., Tidhar, D. (2009). An annotation scheme for citation function. In Proceedings of the 7th SIGdial Workshop on Discourse and Dialogue: 80-87.
Jorg, B. (2008). Towards the Nature of Citations. In Poster Proceedings of the 5th International Conference on Formal Ontology in Information Systems.
Teufel, S., Carletta, J., Moens, M. (1999). An annotation scheme for discourse-level argumentation in research articles. In Proceedings of the 9th Conference of the European Chapter of the Association for Computational Linguistics: 110-117.
Moravcsik, M. J., Murugesan, P. (1975). Some Results on the Function and Quality of Citations. In Social Studies of Science, 5 (1): 86-92.
Qazvinian, V., Radev, D. R. (2010). Identifying non-explicit citing sentences for citation-based summarization. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics: 555-564.
Athar, A., Teufel, S. (2012). Detection of implicit citations for sentiment detection. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: 18-26.
Craik, K. J. W. (1967). The nature of explanation. Cambridge University Press. ISBN:<PHONE_NUMBER>
Di Iorio, A., Nuzzolese, A. G., & Peroni, S. (2013). Towards the automatic identification of the nature of citations. In Proceedings of 3rd Workshop on Semantic Publishing (SePublica 2013): 63-74. Retrieved from http://ceur-ws.org/Vol-994/paper-06.pdf (last visited Jan 13, 2013)
Ciancarini, P., Di Iorio, A., Nuzzolese, A. G., Peroni, S., & Vitali, F. (2013). Characterising citations in scholarly articles: an experiment. In Proceedings of 1st International Workshop on Artificial Intelligence and Cognition (AIC 2013): 124-129. Retrieved from http://ceur-ws.org/Vol-1100/paper13.pdf (last visited Jan 13, 2013)
Sauro, J. (2011). A Practical Guide to the System Usability Scale: Background, Benchmarks & Best Practices. CreateSpace. ISBN:<PHONE_NUMBER>
Lewis, J. R., & Sauro, J. (2009). The Factor Structure of the System Usability Scale. In Proceedings of the 1st International Conference on Human Centered Design (HCD 2009): 94-103. DOI: 10.1007/978-3-642-02806-9_12
Strauss, A. Corbin, J. (1998). Basics of Qualitative Research Techniques and Procedures for Developing Grounded Theory (2nd edition). Sage Publications. ISBN:<PHONE_NUMBER>
Brooke, J. (1996). SUS: a “quick and dirty” usability scale. In Usability Evaluation in Industry: 189-194.
Zhu, X., Turney, P., Lemire, D., & Vellino, A. (2014). Measuring academic influence: Not all citations are equal. To appear in Journal of the American Society for Information Science and Technology. Preprint available at http://lemire.me/fr/documents/publications/citationjasist2013.pdf (last visited Jan 13, 2013)
Shotton, D. (2009). Semantic publishing: the coming revolution in scientific journal publishing. Learned Publishing, 22(2): 85-94. DOI: 10.1087/2009202
Hornsey, M. J., Robson, E., Smith, J., Esposo, S., Sutton, R. M. (2008). Sugaring the Pill: Assessing Rhetorical Strategies Designed to Minimize Defensive Reactions to Group Criticism: Strategies to Soften Criticism. Human Communication Research, 34(1): 70-98. DOI: 10.1111/j.1468-2958.2007.00314.x
MacRoberts, M. H., MacRoberts, B. R. (1984). The Negational Reference: Or the Art of Dissembling. Social Studies of Science, 14(1): 91-94
Biemann, C. (2006). Chinese whispers: an efficient graph clustering algorithm and its application to natural language processing problems. In Proceedings of the 1st Workshop on Graph Based Methods for Natural Language Processing (TextGraph 2006): 73-80
Manning, C. D., & Schütze, H. (1999). Foundations of statistical natural language processing. MIT Press. ISBN:<PHONE_NUMBER>
Osborne, F., & Motta, E. (2012). Mining Semantic Relations between Research Areas. In Proceedings of the 11th International Semantic Web Conference (ISWC 2012): 410-426. DOI: 10.1007/978-3-642-35176-1_26
Kauppinen, T., Baglatzi, A., & Keßler, C. (2013). Linked Science: interconnecting scientific assets. In Data Intensive Science. CRC Press.
Gil, Y., Ratnakar, V., & Hanson, P. C. (2012). Organic data publishing: a novel approach to scientific data sharing. In Proceedings of the 2nd International Workshop on Linked Science (LISC 2012). http://ceur-ws.org/Vol-951/paper1.pdf (last visited Jan 13, 2013)
Shotton, D. (2013). Publishing: Open citations. Nature, 502 (7471): 295-297. DOI: 10.1038/502295a
CiTO: http://purl.org/spar/cito.
CiteULike homepage: http://www.citeulike.org.
Open Linked Data from The Open University: http://data.open.ac.uk.
CiTO Reference Annotation Tools for Google Chrome: https://chrome.google.com/webstore/detail/annotate-journal-citation/geajighoohelnjnhfmhbcaddbcgcbphn.
Link to Link Wordpress plugin: http://wordpress.org/plugins/link-to-link/.
PenSoft Publishers homepage: http://www.pensoft.net/.
CiTalO homepage: http://wit.istc.cnr.it:8080/tools/citalo.
ACL Anthology Network: http://clair.eecs.umich.edu/aan/index.php.
The context window of a citation is a chain of sentences implicitly referring to the citation itself, which usually starts from the citation sentence and involves few more subsequent sentences where that citation is still implicit .
This set of beliefs corresponds to the a user's mental model.
Proceedings of Balisage 2011: http://balisage.net/Proceedings/vol7/cover.html.
It is available for review purposes at http://www.cs.unibo.it/~nuzzoles/cito_1/?user=reviewer.
They are available at http://www.cs.unibo.it/~nuzzoles/cito_2/materials/cito_props.html.
R project for statistical computing: http://www.r-project.org/.
All the data collected and related material are available online at http://www.essepuntato.it/2014/eswc/test.
The authors of CiTO decided on purpose to avoid a taxonomical organisation, since they thought it could be difficult to reach a global agreement.


|
SuanShu, a Java numerical and statistical library | |||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object com.numericalmethod.suanshu.optimization.unconstrained.steepestdescent.SteepestDescent.LineSearch
public abstract class SteepestDescent.LineSearch
A steepest-descent method, in each iteration, searches along a direction to find the next best minimizer along a direction. We want to find out how much the minimizer moves in this direction.
By making LineSearch an inner class,
its methods will have access to the members in SteepestDescent, e.g., f
.
This is to avoid storing the same datum in multiple places, hence no inconsistency problem.
For the very same reason, this class cannot be made static
in order to access the non-static members in SteepestDescent, e.g., f
.
TODO:
Consequently, (a problem?) the member storing an instance of LineSearch,
e.g., lsearch
,
cannot be passed in the constructors.
A non-static inner class cannot be created before the enclosing class is created.
| Field Summary | |
|---|---|
protected Vector |
dk
the line search direction at the k-th iteration |
protected Vector |
gk
the gradient evaluated at xk |
RntoMatrix |
H
the Hessian function, H(x) |
protected Matrix |
Hk
the Hessian evaluated at xk |
| Constructor Summary | |
|---|---|
protected |
SteepestDescent.LineSearch(RntoMatrix H)
Construct a line search instance. |
| Method Summary | |
|---|---|
protected abstract double |
minimize(Vector xk)
Perform a line search to minimize a function, starting from xk . |
protected void |
reset()
Reset the transient states of the instance. |
| Methods inherited from class java.lang.Object |
|---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Field Detail |
|---|
public final RntoMatrix H
H(x)
protected Vector gk
xk
protected Matrix Hk
xk
protected Vector dk
| Constructor Detail |
|---|
protected SteepestDescent.LineSearch(RntoMatrix H)
H
- the function to compute the Hessian at x
| Method Detail |
|---|
protected void reset()
protected abstract double minimize(Vector xk)
xk
.
xk
- the current minimizer
ak+1
|
SuanShu, a Java numerical and statistical library | |||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |


I guess I just coded enough for this night... So here's a little wallpaper, for the Mac addicts around here.
Remembers me the good old time of Mac OS 8.5, on my iMac G3! Enjoy...


| User ccc is Job centeric |
Based on the first graph if you select a filter criteria as Searches done is less than 5 you will see that all the languages of User ccc has been displayed. Hence The users data can be discarder for academic purposes |
| User bbb is ideal for Academic centric research |
Based on the first graph if you select a filter criteria as Jobs done is less than 5 you will see that all the languages of User ccc has been filtered in. Hence The users data is ideal as he has a wide range of Programming interest. On clicking one of users bubble we can see that he has acted upon a wide array of questions |
| Python and Java are trending languages |
Based on the first graph if you select a filter criteria as Searches done are atleast 10 you will see that most users are intersted a lot in python and java hence these languages can be tracked as trending and users should be suggested posts regarding these languages. This graph is extremely usefull in finding outliers. |
| Outliers can easily be identified |
As seen in graph 1 the Graph is very similar to a clustering model and hence we can easily remove the outliers |


AppRun is a JavaScript library for building reliable, high-performance web applications using the Elm-inspired architecture, events, and components.
All the Ways to Make a Web Component - May 2021 Update compares the coding style, bundle size, and performance of 55 different ways to make a Web Component. It put AppRun on the top 1/3 of the list of bundle size and performance.
AppRun is distributed on npm.
npm install apprun
You can also load AppRun directly from the unpkg.com CDN:
<script src="https://unpkg.com/apprun/dist/apprun-html.js"></script>
Or use it as ES module from unpkg.com:
<script type="module">
import { app, Component } from 'https://unpkg.com/apprun/esm/apprun-html?module';
</script>
You can get started with AppRun Docs and the AppRun Playground.
Use the AppRun CLI to initialize an esbuild configured project:
npx apprun --init --spa --esbuild
Use the AppRun CLI to initialize a TypeScript and webpack configured project:
npx apprun --init --spa
To initialize a project that targets ES5, use the AppRun CLI with the --es5 flag:
npx apprun --init --spa --es5
AppRun now has a dev server. It is base on the live-server and supports ES Modules. apprun-dev-server is install by AppRun CLI when using esbuild. Or you can use it directly.
npx apprun-dev-server
See the announcement: A Dev Server Supports ESM
To use the AppRun dev-tools, include the dev-tools script.
<script src="https://unpkg.com/apprun/dist/apprun-dev-tools.js"></script>
See the annoucement: AppRun Dev Tools
AppRun Dev Tools connects to the Redux DevTools Extension. To use the dev-tools, install the Redux DevTools Extension. You can monitor the events and states.
You can launch the webpack dev-server and the demo app from the demo folder with the following npm commands:
npm install
npm start
You can run the unit tests from the tests folder.
npm test
Unit tests can serve as functional specifications.
Finally, to build optimized js files to the dist folder, just run:
npm run build
Have fun and send pull requests.
AppRun is an MIT-licensed open source project. Please consider supporting the project on Patreon. 👍❤🙏
MIT
Copyright (c) 2015-2021 Yiyi Sun


{% extends "base.html" %} {% block style_block %} {% endblock %} {% block content %}
Please consider your most frequently visited online apparel shop when evaluating the website qualities or you may try Gap.com - To see the site, just click on Gap.com. A new window or tab will pop out.
1=strongly disagree and 7=strongly agree
1=least important and 7=most important
Tasks
In order to get a refreshed feeling of the site, imagine you are searching for an apparel item as gift for your friend's birthday and add it to the shopping basket before you start the questionnaire.
To get more information about the quality being evaluated, please hover over the question and you will see more explanation.


[SYMBOL]
[A]
[B]
[C]
[D]
[E]
[F]
[G]
[H]
[I]
[J]
[K]
[L]
[M]
[N]
[O]
[P]
[Q]
[R]
[S]
[T]
[U]
[V]
[W]
[X]
[Y]
[Z]
w command
2nd
w3m command-line browser
Warden, Brett
Warner, Joe
web browsers and Java applets
web information, fetching
web servers allowing unauthorized hosts to access
consolidating logs for
optimizing
WebStart mechanism
WEP (Wireless Encryption Protocol)
multiple NIC configurations
2nd
whatis command
2nd
whatis database, creating
whereis command
which command
who command
window managers screen
multitasking with
showcasing, using eesh utility
Windows using Access Control Lists with
scanning computers for viruses
wiping disks clean before upgrading
Wireless Encryption Protocol (WEP)
multiple NIC configurations
2nd
wireless networks securing with PF
using multiple NIC configurations
words, finding
worms, fighting with Intrusion Detection Systems
wsmoused, shutting down servers using
|


Welcome to Michigan State Underwater Hockey!!
We are looking for people that want to learn what the sport of underwater hockey is and have a good time in doing so.
Practice Schedule
Tuesday: 7:30-9:00 P.M. (IM Circle Pool)
Thursday: 7:30-9:00 P.M. (IM Circle Pool)


Silvia Chyou is currently a first-year master student studying Computer Science in University of California, San Diego (UCSD). She received her bachelor’s degree in Computer Science and Information Engineering at National Taiwan University (NTU). Her area of interests are human-computer interaction and multimedia. The past works she had done includes developing interactive systems, being UI/UX engineer on front-end web applications, and building prototypes of tangible interfaces.


Dublin’s Siobhán McGrath named Players’ Player of the Year
All-Ireland champions lead the way with seven women’s football All Star awards
Dublin’s Siobhán McGrath was named as the 2019 TG4 Senior Players’ Player of the Year at the annual All Star banquet on Saturday night.
McGrath was superb throughout the season as Dublin went on to claim a third successive All-Ireland Senior Championship title in front of a record 56,114 attendance at Croke Park on September 15th.
Thomas Davis star McGrath (31), also claimed a fourth All Star award on the night, following on from previous successes in 2009, 2010 and 2018.
McGrath was the popular choice of her peers to claim the Senior Players’ Player of the Year award, beating off stiff competition from Dublin team-mate Sinéad Goldrick and Galway’s Louise Ward, who also won All Stars on the night.
It’s the third successive year that a Dublin player has won the Players’ Player of the Year award, following wins for Noelle Healy in 2017 and captain Sinéad Aherne in 2018.
Dublin led the way with seven representatives on the 2019 Women’s All Star team. Dublin’s haul matches their all-time best which they also achieved last year.
Goldrick won a seventh All Star in eight years, and there was a fifth award for team-mate Lyndsey Davey, who was Player of the Match against Galway in the All-Ireland Final. Another Dublin star, Carla Rowe, claimed a third award.
Twin sisters
Galway defender Sinéad Burke and Cork forward Orla Finn, who was joint top scorer in the championship, both collected second awards. The selection was also noteworthy for the presence of Galway twin sisters Nicola and Louise Ward.
Nicola and Louise are the first set of twin sisters named in an All Star team since Waterford’s O’Ryan twins, Martina and Geraldine, back in 1992.
It’s also the third successive year that sisters have been named on an All Star selection as the Hegartys from Donegal, Ciara and Niamh, were listed on the 2017 team, while the O’Sullivans from Cork, Ciara and Doireann, were honoured last year.
There were nine first-time winners on the team, including Dublin’s Niamh Collins, Olwen Carey and Niamh McEvoy. Monica McGuirk earned the goalkeeping spot for her exploits with Meath, who contested the All-Ireland Intermediate Final, and it was also a first All Star for the Royal County since 2011, when another goalkeeper, Irene Munnelly, was honoured.
The other first-time recipients were Melissa Duggan (Cork), the Ward sisters, Tracey Leonard, the Galway captain, and Mayo’s Rachel Kearns.
In total, four players survived from the 2018 team – Dublin trio Goldrick, McGrath and Davey, and Galway’s Burke.
It was also confirmed on the night that the 2020 TG4 All Star Tour will visit Austin, Texas, in the United States, with the tour scheduled for March 31-April 8.
The Intermediate Players’ Player of the Year award went to Tipperary’s Aishling Moloney. The 21-year-old Cahir star was named as Player of the Match in the All-Ireland Intermediate Final as Tipperary lifted the Mary Quinn Memorial Cup for the second time in three years.
While Fermanagh may have lost the All-Ireland Junior Final against Louth at Croke Park, 18-year-old Eimear Smyth was voted by her peers as the Junior Players’ Player of the Year.
2019 TG4 WOMEN’S FOOTBALL ALL STAR TEAM:
1. Monica McGuirk (Meath) - 1st award
2. Sinéad Burke (Galway) - 2nd award
3. Niamh Collins (Dublin) - 1st award
4. Melissa Duggan (Cork) - 1st award
5. Nicola Ward (Galway) - 1st award
6. Sinéad Goldrick (Dublin) - 7th award
7. Olwen Carey (Dublin) - 1st award
8. Louise Ward (Galway) - 1st award
9. Siobhán McGrath (Dublin) - 4th award
10. Carla Rowe (Dublin) - 3rd award
11. Niamh McEvoy (Dublin) - 1st award
12. Lyndsey Davey (Dublin) - 5th award
13. Tracey Leonard (Galway) - 1st award
14. Rachel Kearns (Mayo) - 1st award
15. Orla Finn (Cork) - 2nd award


The start of something new. The programming club of IIT Indore has been aiming for this since years, trying to build communities.
This is a small step towards the milestone. We have created a slack workspace with a dedicated channel
for each domain of development. These are more like interest groups, a place for people who share
common interests to discuss and share knowledge. A place where everyone can be a mentor or a mentee
and help each out and rise to glory!
Perks of being in the community? Well, you will be the first to be informed about all the upcoming
hackathon and Dev meet-ups(yes, we are planning that too!). Also, all the projects that the programming
club gets from faculties, and other things that we plan to do. Will first be put up on it.
This is an open group, all of you are welcome to join it. Click the button below to proceed!


Dela677.github.io
PDF Data/Spec Sheet - Vicon Industries.
Vicon V962B-IR312M - SourceSecurity.com. Get in-depth information on Vicon V960B-IR312 IP camera including detailed technical specifications. Besides, view the entire catalog of Vicon V960B-IR312 IP cameras Vicon V960B-IR312 IP camera with specifications of other products from our extensive catalog from leading manufacturers of IP cameras Source any electronic security product from over 20,000 products in the database, make sales. Vicon V960B-IR312 - SourceSecurity.com. This item has been discontinued and is unavailable for purchase. Contact an Anixter representative for possible alternatives.
5 IP Kameras und Komponenten V960-N IP Tag/Nacht Kamera Vicon IP Kameras sind mit Megapixel und SD (Standard Definition) Auflösung erhältlich. Die Kameras können direkt an ein IP Netzwerk angeschlossen werden, und erlauben die Betrachtung und Steuerung per Fernzugriff über eine Video Management Software.
Vicon Deutschland GmbH. Firmenprofil - PDF - docplayer.org. PDF Data/Spec Sheet V960B Series Network Bullet IR Camera. Vicon Camera Encoder Software Downloads Surveyor MKII and V940 Series Software Downloads V942/V944 and V945 Series v4.11.201805281611 and v4.21.201805281611 are available in two formats, a zip file for upgrading through the camera web browser or a bin file for upgrading multiple cameras through the IPCConfig. Vicon - Video Management Solutions and Security Access. View and Download Vicon V962B-IR922M installation manual online. Network Bullet Cameras. V962B-IR922M Security Camera pdf manual download. Also for: V962b-ir312m, V961b-ir312m, V960b-ir312, V960b series. Vicon V962B-IR922M - SourceSecurity.com.
Vicon s V960B Series of HD IP bullet cameras are designed for demanding security installations. It offers a number of fixed network camera versions that deliver crisp clear images to fit any installation. Vicon delivers flexible, end-to-end solutions to meet even the most complex security challenges. Vicon s end-to-end security surveillance solutions deliver simple usability that s as impressive as their powerful capabilities, delivering high-performance, easy-to-deploy solutions. Additional info Vicon s V960B Series of HD IP Bullet Cameras is designed for demanding security installations. It offers a number of fixed network camera versions that deliver crisp clear images to fit any installation need. Vicon V961B-IR312M megapixel true day / night bullet IR CCTV camera Vicon. V960B-IR312 - VICON - IP Bullet Anixter Canada. VICON V962B-IR922M INSTALLATION MANUAL Pdf Download.
Vicon Camera Software Downloads - Vicon Industries.


Classes | |
| class | DownloadProcessMessageContent |
| class | FileNameMessageContent |
| interface | IMessage |
| Defines a Sent event that consumers of a view model can subscribe to in order to listen to the view model's message. More... | |
| class | Message |
| Conveys a message from a view model to a consumer (typically, a view) that has subscribed to the Sent event. More... | |
| class | MessageArgs |
| class | MessageArgsBase |
| class | MessageContent |
| Simple object that encapsulates a boolean value; to be used in MVVM interaction with MessageArgs. More... | |
| class | ProcessMessageContent |
| Holds information about percent completion of a process and defines events that occur when the process is finished. More... | |
| class | StringMessageContent |
| Encapsulates a string message that is part of the content that a view model sends to a consumer (view, test, ...) in a Message. More... | |
| class | SuppressibleMessageContent |
| class | ViewModelMessageContent |
| Message content that holds a reference to a view model. More... | |