|
| Local Methods | ||
| Managed Object Types | Data Object Types | All Properties | All Methods |
Properties
| Name | Type | Description |
|---|---|---|
| None |
Methods
| Methods defined in this Managed Object |
|---|
| QuerySyncingVsanObjectsSummary, VosQueryVsanObjectInformation, VosSetVsanObjectPolicy, VsanDeleteObjects_Task, VsanQueryInaccessibleVmSwapObjects, VsanQueryObjectIdentities |
| Privileges | Required on |
|---|---|
| System.Read | ClusterComputeResource |
Parameters
| Name | Type | Description |
|---|---|---|
| _this | ManagedObjectReference | A reference to the VsanObjectSystem used to make the method call. |
| cluster | ManagedObjectReference
to a ComputeResource |
vSAN cluster
|
| syncingObjectFilter* | VsanSyncingObjectFilter |
Since vSAN API 6.8.7 |
Return Value
| Type | Description |
|---|---|
| VsanHostVsanObjectSyncQueryResult | The space used for data protection by objects from this cluster at the specified remote target locations. The protection is no longer active for these objects. |
Faults
| Type | Description |
|---|---|
| NotSupported | |
| RuntimeFault | Thrown if any type of runtime fault is thrown that is not covered by the other faults; for example, a communication error. |
Events
| Type | |
|---|---|
| None |
| Privileges | Required on |
|---|---|
| System.Read | ClusterComputeResource |
Parameters
| Name | Type | Description |
|---|---|---|
| _this | ManagedObjectReference | A reference to the VsanObjectSystem used to make the method call. |
| cluster* | ManagedObjectReference
to a ComputeResource |
vSAN cluster. Ignored if called against host.
Since vSphere API 5.5 |
| vsanObjectQuerySpecs | VsanObjectQuerySpec[] |
vSAN object query spec list.
|
Return Value
| Type | Description |
|---|---|
| VsanObjectInformation[] | Object information list. |
Faults
| Type | Description |
|---|---|
| FileNotFound | |
| FileNotWritable | |
| InvalidArgument | |
| NotSupported | |
| RuntimeFault | Thrown if any type of runtime fault is thrown that is not covered by the other faults; for example, a communication error. |
Events
| Type | |
|---|---|
| None |
VirtualMachineEmptyProfileSpec
means to use the empty vSAN
policy. This is not the default policy, but a policy where all fields
have default values.VirtualMachineDefinedProfileSpec
where profileId is set, in
which case this profileId will be looked up in SPBM for the detailed
policy information.VirtualMachineDefinedProfileSpec
where profileId is an empty
string and instead the profileData is set for extensionKey
'com.vmware.vim.sps'. In this case the objectData field can be either the
vSAN expression format, or a SPBM XML string. If no profile is
supplied, and the call is executed against vCenter, then SPBM will be
consulted for the vSAN datastore's default profile.| Privileges | Required on |
|---|---|
| Global.Settings | RootFolder |
Parameters
| Name | Type | Description |
|---|---|---|
| _this | ManagedObjectReference | A reference to the VsanObjectSystem used to make the method call. |
| cluster* | ManagedObjectReference
to a ComputeResource |
vSAN cluster. Ignored if called against host.
Since vSphere API 5.5 |
| vsanObjectUuid | xsd:string |
vSAN object UUID.
|
| profile* | VirtualMachineProfileSpec |
See above description for all possible options.
Since vSphere API 5.5 |
Return Value
| Type | Description |
|---|---|
| xsd:boolean | True when the operation successfully, False when it is failed and there is no exception raised. |
Faults
| Type | Description |
|---|---|
| FileNotFound | |
| FileNotWritable | |
| InvalidArgument | |
| NotSupported | |
| RuntimeFault | Thrown if any type of runtime fault is thrown that is not covered by the other faults; for example, a communication error. |
Events
| Type | |
|---|---|
| None |
| Privileges | Required on |
|---|---|
| Host.Config.Storage | ClusterComputeResource |
Parameters
| Name | Type | Description |
|---|---|---|
| _this | ManagedObjectReference | A reference to the VsanObjectSystem used to make the method call. |
| cluster* | ManagedObjectReference
to a ComputeResource |
vSAN cluster. Ignored if called against host.
Since vSAN API vSAN 6.7U1 |
| objUuids | xsd:string[] |
The vSAN object UUIDs to delete.
|
| force* | xsd:boolean |
The force flag, inaccessible objects can't be deleted without
this flag. WARNING: it's at your own risk.
Since vSAN API vSAN 6.7U1 |
Return Value
| Type | Description |
|---|---|
| ManagedObjectReference
to a Task | vim.Task |
Faults
| Type | Description |
|---|---|
| InvalidArgument | |
| RuntimeFault | Thrown if any type of runtime fault is thrown that is not covered by the other faults; for example, a communication error. |
| VsanFault |
Events
| Type | |
|---|---|
| None |
| Privileges | Required on |
|---|---|
| System.Read | ClusterComputeResource |
Parameters
| Name | Type | Description |
|---|---|---|
| _this | ManagedObjectReference | A reference to the VsanObjectSystem used to make the method call. |
| cluster* | ManagedObjectReference
to a ComputeResource |
vSAN cluster. Ignored if called against host.
Since vSAN API vSAN 6.7U1 |
Return Value
| Type | Description |
|---|---|
| xsd:string[] | list of inaccessible vmswap objects. |
Faults
| Type | Description |
|---|---|
| NotSupported | |
| RuntimeFault | Thrown if any type of runtime fault is thrown that is not covered by the other faults; for example, a communication error. |
Events
| Type | |
|---|---|
| None |
| Privileges | Required on |
|---|---|
| System.Read | ClusterComputeResource |
Parameters
| Name | Type | Description |
|---|---|---|
| _this | ManagedObjectReference | A reference to the VsanObjectSystem used to make the method call. |
| cluster* | ManagedObjectReference
to a ComputeResource |
vSAN cluster. Ignored if called against host.
Since vSphere API 5.5 |
| objUuids* | xsd:string[] |
The vSAN object UUIDs for querying. Unset to query
all of vSAN objects
Since vSphere API 5.5 |
| objTypes* | xsd:string[] |
Querying the vSAN objects with given types, only the object whose
type in this list will be returned. All of available object types
include 'vmswap', 'vdisk', 'namespace', 'vmem', 'stats', 'other',
'iscsiHome', 'iscsiLun', 'iscsiTarget' and 'fileShare'.
Unset to query all of vSAN objects with any type
Since vSphere API 6.5 |
| includeHealth* | xsd:boolean |
If the result includes the object health status.
Default is False
Since vSphere API 5.5 |
| includeObjIdentity* | xsd:boolean |
If the result includes all of the object identity.
Default is false if it's unset
Since vSphere API 5.5 |
| includeSpaceSummary* | xsd:boolean |
If the result includes the vSAN space consumption
summarizing by object type. Currently, it's not
supported to include both of space summary and
object identity in the result. Default is false.
Since vSphere API 5.5 |
Return Value
| Type | Description |
|---|---|
| VsanObjectIdentityAndHealth | Object information structure |
Faults
| Type | Description |
|---|---|
| NotSupported | |
| RuntimeFault | Thrown if any type of runtime fault is thrown that is not covered by the other faults; for example, a communication error. |
| VsanFault |
Events
| Type | |
|---|---|
| None |
| Top of page | Local Methods | ||
| Managed Object Types | Data Object Types | All Properties | All Methods |


Package minesim
|
95%
successful |
Failed tests
- DatabaseHandlerTests. ItemManipulation
- DatabaseHandlerTests. selectItems
- LoginDatabaseHandlerTests. testLoginExistsPartFourWithLogger
- LoginDatabaseHandlerTests. testLoginExistsPartThreeWithLogger
- LoginDatabaseHandlerTests. testLoginExistsPartTwoWithLogger
- LoginDatabaseHandlerTests. testLoginExistsWithLogger
- LoginDatabaseHandlerTests. updateSavedSettingsMultiUser
- LoginDatabaseHandlerTests. updateSavedSettingsSingleUser
Ignored tests
Classes
| Class | Tests | Failures | Ignored | Duration | Success rate |
|---|---|---|---|---|---|
| AchievementTest | 2 | 0 | 0 | 0.169s | 100% |
| AnimationTests | 19 | 0 | 0 | 0.576s | 100% |
| ApparelTests | 2 | 0 | 0 | 0.012s | 100% |
| AutoTaskTests | 1 | 0 | 0 | 0.884s | 100% |
| BuffTest | 7 | 0 | 0 | 0.660s | 100% |
| BuildingTests | 12 | 0 | 0 | 0.515s | 100% |
| ContextAreaHandlerTests | 7 | 0 | 0 | 0.517s | 100% |
| DatabaseHandlerTests | 3 | 2 | 0 | 0.975s | 33% |
| DigTileTests | 1 | 0 | 0 | 1.067s | 100% |
| FrameTests | 8 | 0 | 0 | 0.081s | 100% |
| InventoryTests | 19 | 0 | 0 | 0.146s | 100% |
| ItemApparelTests | 1 | 0 | 0 | 0s | 100% |
| ItemFoodTests | 3 | 0 | 0 | 0.001s | 100% |
| ItemMinedEntityTests | 3 | 0 | 0 | 0.002s | 100% |
| ItemTests | 9 | 0 | 0 | 0.299s | 100% |
| ItemToolTests | 6 | 0 | 0 | 0.002s | 100% |
| ItemWeaponTests | 7 | 0 | 0 | 0.002s | 100% |
| LoginDatabaseHandlerTests | 10 | 6 | 0 | 0.317s | 40% |
| MineSimTests | 1 | 0 | 0 | 0s | 100% |
| MobTests | 5 | 0 | 0 | 0.152s | 100% |
| MusicTests | 20 | 0 | 2 | 11.516s | 100% |
| PeonSelectortests | 5 | 0 | 4 | 0.268s | 100% |
| PeonTests | 15 | 0 | 0 | 0.006s | 100% |
| PickUpItemTests | 2 | 0 | 0 | 0.005s | 100% |
| ResizableCanvasTests | 2 | 0 | 0 | 0.006s | 100% |
| SettingsHandlerTests | 5 | 0 | 0 | 0.005s | 100% |
| SoundTests | 9 | 0 | 0 | 0.244s | 100% |
| SpriteLoaderTests | 8 | 0 | 0 | 0.020s | 100% |
| TransportationTest | 7 | 0 | 0 | 0.026s | 100% |
| WorldTests | 5 | 0 | 0 | 1.115s | 100% |
| ZombieMobTest | 1 | 0 | 0 | 0.001s | 100% |


Welcome to Bootparts Technical Documentation Page template. It was designed to simplify the process of writing technical documentation. The content uses unstyled HTML tags with a straightforward structure.
We don't like messy documentation pages. That's why this one is a simple two column page that is mobile friendly.
Good documentation is not only about the content. How you structure the content also matters. That's why this template has a very simple structure.
Put everything on the left side column. Keep it simple.


The Apache Drill developer community is growing rapidly and currently includes dozens of contributors employed by many organizations, such as MapR, Intuit, Hortonworks, Elastic, LinkedIn, Pentaho, Cisco and the University of Wisconsin-Madison.
We welcome contributions to the project. If you’re interested in contributing, take a look at the contribution guide. You can also ask questions on the drill-dev list.
| Name | **Alias (email is |
|---|---|
| Abdel Hakim Deneche | adeneche |
| Aditya Kishore | adi |
| Abhishek Girish | agirish |
| AnilKumar B | akumarb2010 |
| Aman Sinha | amansinha |
| Arina Ielchiieva | arina |
| Boaz Ben-Zvi | boaz |
| Bridget Bevens | bridgetb |
| Kamesh Bhallamudi | bvskamesh |
| Charles Givre | cgivre |
| Chunhui Shi | cshi |
| Chris Wensel | cwensel |
| Chris Westin | cwestin |
| Ellen Friedman | ellenf |
| German Shegalov | gera |
| Gautam Parai | gparai |
| Grant Ingersoll | gsingers |
| Hanifi Gunes | hg |
| Hanumath Rao Maduri | hmaduri |
| Hsuan-Yi Chu | hsuanyichu |
| Isabel Drost-Fromm | isabel |
| Jacques Nadeau | jacques |
| Jason Frantz | jason |
| Julian Hyde | jhyde |
| Jinfeng Ni | jni |
| Jason Altekruse | json |
| Karthikeyan Manivannan | karthikm |
| Keys Botzum | kbotzum |
| Kris Hahn | krishahn |
| Kunal Khatua | kunal |
| Laurent Goujon | laurent |
| Mehant Baid | mehant |
| Neeraja Rentachintala | neerajar |
| Parth Chandra | parthc |
| Padma Penumarthy | ppadma |
| Paul Rogers | progers |
| Ryan Rawson | rawson |
| Rahul Kumar Challapalli | rkins |
| Salim Achouche | sachouche |
| Steven Phillips | smp |
| Sorabh Hamirwasia | sorabh |
| Srivas | srivas |
| Sudheesh Katkam | sudheesh |
| Ted Dunning | tdunning |
| Timothy Farkas | timothyfarkas |
| Timothy Chen | tnachen |
| Tomer Shiran | tshiran |
| Venki Korukanti | venki |
| Vitalii Diravka | vitalii |
| Vova Vysotskyi | volodymyr |
| Weijie Tong | weijie |


I'm currently a software engineer at Google Photos. I received my PhD in Modeling, Simulation and Visualization Engineering at Old Dominion University in 2021. I'm interested in the areas of computer graphics, computational photography, and machine learning.
Part of the computational photography and image editing team on Google Photos Android. Building and maintaining ML-powered effects used by millions: Magic Eraser, Photo Unblur, Auto-Enhance with Real Tone.
Other projects include redesigning the real-time rendering framework of the image editor, showcasing computational effects in other surfaces of the app, and fast experimentation to increase usage of the editor.
August 2021 - Present
Research on the areas of computer graphics and deep learning.
Leader of the MSVE Demo team.
Teacher assistant of Computer Graphics and Intro To Game Development.
August 2016 - May 2021
Handling all communication and coordination between faculty at ODU and administration at VMASC.
Responsible for digital media, marketing, website development, student coordination, conference handbook publication, and track chair coordination.
January 2017 - May 2021
Research on machine learning techniques to improve Draco: a library for compression and decompression of 3D geometric meshes.
Successfully reduced prediction error by 20-40% with respect to Draco’s parallelogram predictors.
May 2020 - August 2020
Research on machine learning techniques focused on the automatic generation of 3D models.
Development of a pipeline for the parameterization of 3D models into geometric images without loss of information.
May 2019 - August 2019
Virtual Reality development for Samsung Gear VR and Oculus Rift.
Data Processing using Amazon Web Services.
March 2016 - June 2016
Dissertation: Feature Extraction and Design in Deep Learning Models
GPA: 3.98
GPA: 3.98
GPA: 3.91


Harmeet downs Amalraj for title
by Sports BureauBATAM (INDONESIA)Harmeet Desai, seeded fourth, defeated seventh seed A. Amalraj 11-9, 9-11, 11-9, 11-9, 10-12, 11-9 in the final of the ITTF Challenge Indonesia Open table tennis championship here on Sunday.
This is Harmeet’s second international title of the year after the Commonwealth championship in July at Cuttack.
Harmeet said it felt great to win his first title on foreign soil.
“It was a very different feeling. It wasn’t a easy tournament for me as I had tough matches from the first round and I beat quite a few good Asian players,” said the 26-year-old.


--- title: Test \_all_ keywords: fastai sidebar: home_sidebar summary: "Does it matter that \_all_ can't have line breaks?" description: "Does it matter that \_all_ can't have line breaks?" ---
Without line breaks, _all_ to __all__ works great (o:
__all__ = ['_not_included_by_default', '_not_included_by_default2']
# Cell
def _not_included_by_default(): pass
def _not_included_by_default2(): pass
With line breaks, _all_ doesn't get picked up by _re_all_def
(... ([^\n\]]*) # Catching group with anything except a ] or newline
)
__all__ = []
# Cell
def _not_included_by_default(): pass
def _not_included_by_default2(): pass
# Cell
_all_=["_not_included_by_default",
"_not_included_by_default2"]
Note: _re__all__def
uses a different catching group ([^\]]*)
that allows line breaks
_all_
right next to the thing we need to include that's what this nb has been changed to


click to get h1 and url from content.js
show list of dummy articles
Value:
Welcome! Please set your political preference below:
I have no political preference/I do not know my political preference
My political preference is:
Enter


Léxi is an online realtime chat application that allows users to chat securely knowing that their information is not being sold to third parties. Try it now


VTA Hardware Guide¶
We present a top-down overview of the VTA hardware design. This hardware design guide covers VTA hardware at two levels:
An architectural overview of the VTA design and its ISA hardware-software interface.
A micro-architectural overview of the VTA hardware modules, and the micro-code specification for the compute core.
VTA Overview¶
VTA is a generic deep learning accelerator built for fast and efficient dense linear algebra. VTA incorporates a simple RISC-like processor that can perform dense linear algebra operations on rank 1 or 2 tensor registers. In addition the design adopts decoupled access-execute to hide memory access latency.
To a broader extent, VTA can serve as a template deep learning accelerator design for full stack optimization, exposing a generic tensor computation interface to the compiler stack.
The figure above gives a high-level overview of the VTA hardware organization. VTA is composed of four modules that communicate among each other via FIFO queues and local memory blocks (SRAM), to enable task-level pipeline parallelism:
The fetch module takes care of loading an instruction stream from DRAM. It also decodes those instructions to route them into one of three command queues.
The load module takes care of loading input and weight tensors from DRAM into data-specialized on-chip memories.
The compute module performs both dense linear algebra computation with its GEMM core, and general computation with its tensor ALU. It also takes care of loading data from DRAM into the register file, and loading micro-op kernels into the micro-op cache.
The store module stores results produced by the compute core back to DRAM.
HLS Hardware Source Organization¶
The VTA design is currently specified in Vivado HLS C++, which is only supported
by Xilinx toolchains.
The VTA hardware sources are contained under vta/hardware/xilinx/sources
:
vta.cc
contains the definitions for each VTA module, as well as a top level behavioral model for the top-level VTA design.
vta.h
contains type definitions using Xilinxap_int
types, and function prototypes declarations.
In addition preprocessor macros are defined under vta/include/vta/hw_spec.h
.
Much of these macro definitions are derived from the parameters listed in the
vta/config/vta_config.json
file.
The json file is processed by vta/config/vta_config.py
to produce a string of
compile flags that define the preprocessor macros.
That string is used by the makefile in order to set those high-level
parameters in both the HLS hardware synthesis compiler, and the C++
compiler that builds the VTA runtime.
HLS Module Example¶
We show a definition of one of the VTA modules defined in C++:
void fetch(
uint32_t insn_count,
volatile insn_T *insns,
hls::stream<insn_T> &load_queue,
hls::stream<insn_T> &gemm_queue,
hls::stream<insn_T> &store_queue) {
#pragma HLS INTERFACE s_axilite port = insn_count bundle = CONTROL_BUS
#pragma HLS INTERFACE m_axi port = insns offset = slave bundle = ins_port
#pragma HLS INTERFACE axis port = load_queue
#pragma HLS INTERFACE axis port = gemm_queue
#pragma HLS INTERFACE axis port = store_queue
#pragma HLS INTERFACE s_axilite port = return bundle = CONTROL_BUS
INSN_DECODE: for (int pc = 0; pc < insn_count; pc++) {
#pragma HLS PIPELINE II = 1
// Read instruction fields
insn_T insn = insns[pc];
// Do some partial decoding
opcode_T opcode = insn.range(VTA_INSN_MEM_0_1, VTA_INSN_MEM_0_0);
memop_id_T memory_type = insn.range(VTA_INSN_MEM_5_1, VTA_INSN_MEM_5_0);
// Push to appropriate instruction queue
if (opcode == VTA_OPCODE_STORE) {
store_queue.write(insn);
} else if (opcode == VTA_OPCODE_LOAD &&
(memory_type == VTA_MEM_ID_INP || memory_type == VTA_MEM_ID_WGT)) {
load_queue.write(insn);
} else {
gemm_queue.write(insn);
}
}
}
- A few observations on HLS coding:
Parameters: The parameter list of each function, combined with the interface pragmas define the hardware interface exposed by the generated hardware module.
Parameters passed by value indicate a read-only hardware memory-mapped register that the host can write to. This fetch function for instance has an
insn_count
parameter which will be synthesized as a memory mapped register for the host to write to, in order to set the length of a given VTA instruction sequence.Pointer parameters can mean one of two things depending on the interface pragma being used.
When used with a
m_axi
interface pragma, an AXI master interface gets generated to provide DMA access to DRAM.When used with a
bram
interface pragma, a BRAM interface gets generated to expose read and/or write ports to an FPGA block-RAM.
HLS streams being passed by reference combined with the
axis
interface pragma produce FIFO interfaces to the module. Hardware FIFOs provide a useful synchronization mechanism between modules.
Pragmas: Compiler pragmas are essential to define hardware implementation of each module. We list several pragmas used in the VTA design to communicate implementation requirements to the compiler.
HLS INTERFACE
: specifies the interface of the synthesized hardware module.HLS PIPELINE
: defines hardware pipeline performance target by setting an initiation interval goal. When theII == 1
target is set, it tells the compiler that the synthesized hardware pipeline should be able to execute one loop iteration per cycle.HLS DEPENDENCE
: instructs the compiler to ignore certain types of dependence checks in a given loop. Consider a loop body that writes and reads to the same BRAM structure, and needs to achieve an II of 1. The HLS compiler has to assume worst-case scenario, whereby a read is issued to an address that a past write updated the cycle prior: this cannot be achieved given BRAM timing characteristics (it takes at least 2 cycles to see the updated value). Therefore in order to achieve an II of 1, the dependence checks have to be relaxed. Note that when turning this optimization on, it falls onto the software stack to prevent writes followed by reads to the same address.
Note
This reference guide provides a much more in-depth, and complete specification of HLS for the Xilinx 2018.2 toolchains.
Architectural Overview¶
Instruction Set Architecture¶
VTA’s instruction set architecture (ISA) is composed of 4 CISC instructions that have a variable execution latency, two of which execute a micro-coded instruction sequence to perform computation.
The VTA instructions are listed below:
LOAD
instruction: loads a 2D tensor from DRAM into the input buffer, weight buffer, or register file. It can also load a micro-kernel into the micro-op cache. Supports dynamic padding when loading input and weight tiles.GEMM
instruction: performs a micro-op sequence of matrix-matrix multiplications over an input tensor and a weight tensors, and adds the result to a register-file tensor.ALU
instruction: performs a micro-op sequence of matrix-matrix ALU operations over register-file tensor data.STORE
instruction: stores a 2D tensor from the output buffer to DRAM.
The LOAD
instructions are executed by the load and compute modules depending on the store memory buffer location target.
The GEMM
and ALU
instructions are executed by the compute module’s GEMM core and tensor ALU.
Finally, the STORE
instructions are executed by the store module exclusively.
The fields of each instruction is described in the figure below.
The meaning of each field will be further explained in the vta-uarch section.
Note
Note that the VTA ISA changes as VTA’s architectural parameters are modified (i.e. GEMM core shape, data type, memory size etc.), and as a result the ISA does not guarantee compatibility across all variants of VTA. This is acceptable however, since the VTA runtime adapts to parameter changes, and produces binary code tailored for the version of the accelerator that gets generated. This exemplifies the co-design philosophy adopted by the VTA stack which embraces fluidity of the hardware-software interface.
Dataflow Execution¶
VTA relies on dependence FIFO queues between hardware modules to synchronize the execution of concurrent tasks. The figure below shows how a given hardware module can execute concurrently from its producer and consumer modules in a dataflow fashion through the use of dependence FIFO queues, and single-reader/single-writer SRAM buffers. Each module is connected to its consumer and producer via read-after-write (RAW) and write-after-read (WAR) dependence queues.
The pseudo-code above describes how a module executes a given instruction predicated on dependences with other instructions. First, the dependence flags within each instruction are decoded in hardware. If the instruction has an incoming RAW dependences, execution is predicated upon receiving a RAW dependence token from the producer module. Similarly, if the task has an incoming WAR dependence, execution is predicated upon receiving a WAR dependence token from the consumer module. Finally when the task is done, we check for outgoing RAW and WAR dependences, and notify the consumer and producer modules respectively.
Note
Note that the dependence tokens in this scenario are information-less. This is because the instructions executed by each module cannot be reordered by design, as they arrive in FIFO order.
Pipeline Expandability¶
The default VTA design is composed of four modules that describe a 3-stage load-compute-store
task pipeline.
Following the dataflow hardware organization principle, we can extend VTA the pipeline to include more stages.
For example, we can envision separating the tensor ALU from the GEMM core in order to maximize the utilization of the GEMM core.
This would result in a load-gemm-activate-store
task pipeline which closely reflects the TPU design.
Adding more stages has a cost however: it can add storage and extra logic overhead, which is why we opted for a default 3-stage pipeline.
Microarchitectural Overview¶
We describe the modules that compose the VTA design.
The module definitions are contained in vta/hardware/xilinx/sources/vta.cc
.
Fetch Module¶
VTA is programmed by a linear instruction stream. The fetch module is the entry point of VTA to the CPU and is programmed via three memory mapped registers:
The read-write
control
register starts the fetch module, and is read to check for its completion.The write-only
insn_count
register sets the number of instructions to execute.The write-only
insns
register sets the start address of the instruction stream in DRAM.
The CPU prepares the instruction stream in DRAM in a physically-contiguous buffer prepared by the VTA runtime.
When the instruction stream is ready, the CPU writes the start physical address into the insns
register, the length of the instruction stream into the insn_count
register, and asserts the start signal in the control
register.
This procedure starts VTA, which reads in the instruction stream from DRAM via DMA.
Upon accessing the instruction stream, the fetch module partially decodes instructions, and pushes those instructions into command queues that feed into the load, compute, and store modules:
STORE
instructions are pushed to the store command queue to be processed by the store module.GEMM
andALU
instructions are pushed to the compute command queue to be processed by the compute module.LOAD
instructions that describe a load operation of micro-op kernels or register file data are pushed to the compute command queue to be processed by the compute module.LOAD
instructions that describe a load operation of input or weight data are pushed to the load command queue to be processed by the load module.
When one of the command queues becomes full, the fetch module stalls until the queue is not full.
Consequently, the command queues are sized to be deep enough to allow for a wide execution window, and allow multiple tasks to be in flight concurrently across the load-compute-store
pipeline.
Compute Module¶
VTA’s compute module acts as a RISC processor that performs computation on tensor registers rather than scalar registers. Two functional units mutate the register file: the tensor ALU, and the GEMM core.
The compute module executes RISC micro-ops from the micro-op cache. There are two types of compute micro-ops: ALU and GEMM operations. To minimize the footprint of micro-op kernels, while avoiding the need for control-flow instructions such as conditional jumps, the compute module executes micro-op sequences inside a two-level nested loop that computes the location of each tensor register location via an affine function. This compression approach helps reduce the micro-kernel instruction footprint, and applies to both matrix multiplication and 2D convolution, commonly found in neural network operators.
The GEMM core evaluates GEMM instructions, by executing a micro-code sequence in a 2-level nested loop described in the Figure above. The GEMM core can perform one input-weight matrix multiplication per cycle. The dimensions of the single-cycle matrix multiplication defines a hardware tensorization intrinsic which the TVM compiler has to lower a computation schedule onto. This tensorization intrinsic is defined by the dimensions of the input, weight and accumulator tensors. Each data type can have a different integer precision: typically both weight and input types are low-precision (8-bits or less), while the accumulator tensor has a wider type to prevent overflows (32-bits). In order to keep the GEMM core busy, each of the input buffer, weight buffer, and register file have to expose sufficient read/write bandwidth.
The Tensor ALU supports a set of standard operations to implement common activation, normalization, and pooling operators. VTA being a modular design, the range of operators that the Tensor ALU supports can be extended for higher operator coverage, at the expense of higher resource utilization. The Tensor ALU can perform tensor-tensor operations, as well as tensor-scalar operations on an immediate value. The opcode of the tensor ALU, and the immediate value are specified by the high-level CISC instruction. The micro-code in the context of tensor ALU computation only takes care of specifying data access patterns.
Note
In terms of computational throughput, the Tensor ALU does not execute at a rate of one operation per cycle. The limitation comes from the lack of read-ports: since one register file tensor can be read per cycle, the tensor ALU has an initiation interval of at least 2 (i.e. performs at most 1 operation every 2 cycles). In addition, performing a single tensor-tensor operation at once can be expensive especially given that register file types are wide, typically 32-bit integers. As a result, in order to balance the resource utilization footprint of the Tensor ALU with the GEMM core, a tensor-tensor operation is by default performed via vector-vector operations over multiple cycles.
Load and Store Modules¶
The load and store modules perform 2D DMA loads with a strided access pattern from DRAM to SRAM. In addition, the load module can insert 2D padding on the fly, which is useful when blocking 2D convolution. This means that VTA can tile 2D convolution inputs without paying the overhead of re-laying data out in DRAM to insert spatial padding around input and weight tiles.


The Talented Mr Ripley: single white male
These Pacific Heights type stories are usually told from the victims' point of view, which serves to make the 'villain' more threatening, as he could pop into any scene. In The Talented Mr. Ripley, director Anthony Minghella (working from Patricia Highsmith's novel, the first in a series of Ripley-novels) takes the alternate route, tells the story from the 'villain's' point of view. Meaning he's in just about every scene, so what 'pops up' to surprise us from time to time is his sociopathic nature.  But that's his talent--the human chameleon. And, as all chameleons (and secret agents, a similar breed) must, he has no past, no history; there's no real 'explanation' for his actions, and barely any hints. Meaning of course he's the ideal blank slate upon which to impress an identity, any identity. All he needs is a mark.
Highsmith supplies one via a little Henry James/Ambassadors shuffle: Ripley (Matt Damon) bluffs his way into an assignment to travel to Italy and retrieve a rich man's son, Dickie Greenleaf, (Jude Law) living in some particularly Edenic niche of Italy with Marge, (Gwyneth Paltrow) his girlfriend. So yes, it's a guy, a girl, and a garden spot. All we need now is the snake: enter Ripley, ingratiating himself into their confidences, living with them, studying them. Initially it's all about adoration, but that quickly escalates into jealousy, and from jealousy it's an easy step up to the conclusion that they--or, Dickie--isn't really deserving of this decadent, expatriate lifestyle. Or, at least isn't as deserving, which, to Ripley--already operating under the Raskolnikov moral code (if I can do it, I should)--serves as all the moral compulsion/permission he needs. When the circumstances permit, and as the trailer tells us, he becomes Dickie, assumes Dickie's identity, which is where The Talented Mr. Ripley really gets fun.
We did love Jack Tripper's antics after all--always pretending to be one, two, or more people at once. Ripley's doing essentially the same thing, only now, when the coincidences start piling up against him, he simply kills his way out. Anything but get caught. And he does have some Flashman (George MacDonald Fraser's 'Ripley') type luck, but not too much. Meaning it's all credible, more or less, even as the body count rises. Interestingly, too, Minghella, instead of having his 'hero' (Ripley) be flat out bad, draws him as strangely driven, each time torturing himself for what he 'has' to do, etc. Which is a nice touch, allows us a little sympathy for an otherwise unsympathetic character, enough that we even want him to get out of each scrape alive, by his wits alone (per Bond). It's impressive, or, impressively written, and never quite gets stale or overly predictable. The last twenty minutes even tease us with a Clay Pigeons ending, by introducing a private investigator (Philip Baker Hall, still Seinfeld's library cop) and a run of particularly bad luck for Ripley. For him, though, there's always an answer, a proven solution to anything: just start killing. And the thing is, we kind of want him to. His most attractive quality, after all, is his sociopathic nature, so the more it surfaces, the better. With Silence of the Lambs, what we really wanted was more Hannibal. In The Talented Mr. Ripley, we get it.
(c) 1999 Stephen Graham Jones, http://www.cinemuck.com
The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


ABOUT
.
CHECKOUT/VIEW CART
BRANDS
Angeldustrial
DISTRO.Y
The Void Above
Zazzi
CATEGORIES
Womens
Apparel
Sale!
Knitwear
Dresses
Mens / Unisex
Sale!
Accessories
New Items
SUPPORT
About
Policies
FAQ's
Contact Us
Order Status
LINKS
WHOLESALE
BLOG
Links
Here are some links around the web of stuff we love and admire.
Email us at
s
upport
@
dis
t
roy
.
c
om if you have a link we should check out or are interested in exchanging links.
Our Sites:
TeaCurran.com
GrilledCheese.com
Fontlinks.com
PunkHost.com
Reading:
12oz Prophet
Beautiful Decay
Flaunt
Nylon
While You Were Sleeping
Art & Design:
The Barbarian Group
Full Time Artists
Girl Chaos
Graffiti Fonts .com
Technologikill
Friends:
99th Mind
Angeldustrial
Hubba Hubba
Delete Yourself
Lady Pussycat
Latex Lily
Scott Lanes
Seze
Stel's
Music:
Coil
End
Forced Exposure
Honey Pump
Infrasound
Mahi Mahi
Tiger Beat6
Clothing & Accessories:
My Paper Crane
Neuty
Safe Clothes
Slow Shirts
Nemo Leather
©2023 DISTRO.Y /
Approaching Pi, Inc.
Please choose a variation and size


{* this is a fragment template of the main question set *} {* removed useless comment tags by Maikel *} {if 1==0} {/if}
{*[th]By filling this form you agree that our company can work with your personal data[/th]*} {*[th]You need to agree with processing your personal data[/th]*}


Smacc: a Compiler-Compiler
Vocabulary
This chapter defines some vocabulary used by Smacc.
Reference Example
Let us take the following grammar.
Metagrammar structure
SmaCC grammars are written in EBNF format (Extended Backus-Naur Form) with a syntax ressembling closely to the one of GNU Bison. A grammar is composed of:
- Scanner rules: they define tokens to recognize in the input stream through regex,
- Parser rules: they define the production rules of your grammar,
- Directives: they are additionnal information for the parsing or for the AST generation.
Note that you can also find the metagrammar of SmaCC described in itself in the SmaCCDefinitionParser
.
Elements
Production rule
The following expressions define two production rules.
A production rule is defined by a left hand side and several alternatives.
- Here the first production rule has two alternatives.
- While the seconde production rule has only one.
An alternative can be composed of any variation of:
- non terminals often starting with uppercase
- scanner tokens
- keywords (delimited by
"
)
In addition, you can use the single curly braces {}
to define an arbitrary semantic action or the double curly braces {{}}
to create an AST node instead.
Non terminals and tokens can be annotated with variable names (delimited by '
) that will be the instance variable names of the AST node.
Tokens
Tokens are identified by the scanner. A token specification is composed of a token name and a token regular expression.
The following token specification describes a number. It starts with one or more digits, possibly followed by a decimal point with zero or more digits after it. The scanner definition for this token is:
Let's go over each part:
<number>
- Names the token identified by the expression. The name inside the <> must be a legal Pharo variable name.
:
- Separates the name of the token from the token's definition.
[0-9]
- Matches any single character in the range
'0'
to'9'
(a digit). We could also use\d
or<isDigit>
as these also match digits. +
- Matches the previous expression one or more times. In this case, we are matching one or more digits.
( ... )
- Groups subexpressions. In this case we are grouping the decimal point and the numbers following the decimal point.
\.
- Matches the '.' character (. has a special meaning in regular expressions, quotes it).
*
- Matches the previous expression zero or more times.
?
- Matches the previous expression zero or one time (i.e., it is optional).
;
- Terminates a token specification.
Keywords
Keywords are defined in the production and delimited by "
.
Keywords are only defined through static strings, regular expressions cannot be used.
In the following example, "+"
and "-"
are considered keywords.
Non Terminal
In the production rule Expression 'left' "+" 'operator' Expression 'right'
, Expression is a non terminal.
Variables
Variables give name to one element of a production. For example
- 'left' and 'right' denote the first and second expression of the alternative.
- 'operator' denotes the caret token.


Ryan Kent: 'Hamilton result proves Rangers are up for title scrap'
Ryan Kent insists Rangers are right up for the title fight after beating a Hamilton side he believes would have flattened them last season.

He then rolled home again in stoppage time to put the seal on a 3-1 win but admits the Ibrox side failed to produce their best during a sloppy second-half.
However, the fact the Light Blues ground out the victory that sees them keep pace with title rivals Celtic at the top of the table is proof enough for Kent that his side will be in the hunt for the Ladbrokes Premiership crown all the way to May.
Kent – whose side saw Ryan Jack's early opener cancelled out by Lewis Smith's header for Hamilton – said: "There's always one eye on what's going on across the city but we can only control what we're doing and what's going on at our end.
"But we're always up for the challenge – we were up for the challenge last season but we fell short and we're here to put it right this season.
"The winning mentality has changed and players believe more in themselves as you can see from the performances of everyone who has stepped up another level this season.
"Coming off the pitch we probably weren't happy with our performance but we're quite pleased with the result because a year ago who knows what would have happened in a game like that?
"If we go into the game with the right mentality and treat the opposition with respect then it's down to us and our performances."
Gers are unlikely to get away with being so off the pace when they travel to Rotterdam on Thursday to face Feyenoord, where a win will seal progress to the last 32 of the Europa League.
But Kent has no doubt Rangers will respond positively at De Kuip.
He said: "In the bigger games we get the best out of ourselves and maybe being underdogs helps that.
"Our performances in the Europa League so far have been excellent and not just with the ball but without it as well.
"It's an important run of games coming up and hopefully there will be a lot of happy smiles after it but we need to take one game at a time and make sure we're putting in 100 per cent in every game and hopefully it will all come good."
body check tags ::


News,
5/10/2002
New product release, Now with more update and more bug fix.
5/10/2002
New product release, Now with more update and more bug fix.
Contents,
Your contents goes here...
Footer information goes below here...
Site designed by
Template empire


risk production livestock health zoonotic collab: {border-style human px;} #ffffff;} disease emergence research animal United Health associated economic pathogen pathogens policy transmission [accessed analysis animals global interdisciplinary public June new Livestock development diseases example, likely systems Sci food local px; range social {font-family agricultural avian different emerging font-size framework genetic government humans large options political trade use Biol However, London, Organization Research account affect changes double; drivers evolution factors farm including infectious influenza integration intensification market natural needed potential provide spread Agriculture Kingdom[], London Med approach border-color border-width conceptual conditions contact control data diverse domestic emergence, equilibrium identify implications increase influence level levels management management. population poultry practices producers production, reduce requires risks solid; understanding units units, ways wild {font-style {font-weight Environmental Medicine, Nipah Policy Public Rome:Food Soc Tropical World antibiotic bold complex computable courier; ecological effects em; environment environments epidemiological future general growing highly host important increasingly industrial insights intensive interactions international lack modeling models novel pig populations process pt;} risk. scenarios studies support understand waste


The Quota for Exercise of Parliamentary Activity says that meal expenses can be reimbursed just for the politician, excluding guests and assistants. Creating a feature with information of "traveled speed" (i.e. too many meals in distant cities, in a short period of time) from last meal can help us detect anomalies compared to other expenses.
Since we don't have in structured data the time of the expense, we want to anylize the group of expenses made in the same day.
import pandas as pd
import numpy as np
reimbursements = pd.read_csv('../data/2016-11-19-reimbursements.xz',
dtype={'cnpj_cpf': np.str},
low_memory=False)
reimbursements.iloc[0]
reimbursements = reimbursements[reimbursements['subquota_description'] == 'Congressperson meal']
reimbursements.shape
reimbursements['issue_date'] = pd.to_datetime(reimbursements['issue_date'], errors='coerce')
reimbursements.sort_values('issue_date', inplace=True)
companies = pd.read_csv('../data/2016-09-03-companies.xz', low_memory=False)
companies.shape
companies.iloc[0]
companies['cnpj'] = companies['cnpj'].str.replace(r'[\.\/\-]', '')
dataset = pd.merge(reimbursements, companies, left_on='cnpj_cpf', right_on='cnpj')
dataset.shape
dataset.iloc[0]
Remove party leaderships from the dataset before calculating the ranking.
dataset = dataset[dataset['congressperson_id'].notnull()]
dataset.shape
And also remove companies mistakenly geolocated outside of Brazil.
is_in_brazil = '(-73.992222 < longitude < -34.7916667) & (-33.742222 < latitude < 5.2722222)'
dataset = dataset.query(is_in_brazil)
dataset.shape
# keys = ['applicant_id', 'issue_date']
keys = ['congressperson_name', 'issue_date']
aggregation = dataset.groupby(keys)['total_net_value']. \
agg({'sum': np.sum, 'expenses': len, 'mean': np.mean})
aggregation['expenses'] = aggregation['expenses'].astype(np.int)
aggregation.sort_values(['expenses', 'sum'], ascending=[False, False]).head(10)
len(aggregation[aggregation['expenses'] > 7])
keys = ['congressperson_name', 'issue_date']
cities = dataset.groupby(keys)['city']. \
agg({'city': lambda x: len(set(x)),
'city_list': lambda x: ','.join(set(x))}
).sort_values('city', ascending=False)
cities.head()
cities[cities['city'] >= 4].shape
Would be helpful for our analysis to have a new column containing the traveled distance in this given day.
from geopy.distance import vincenty as distance
from IPython.display import display
x = dataset.iloc[0]
display(x[['cnpj', 'city', 'state_y']])
distance(x[['latitude', 'longitude']],
x[['latitude', 'longitude']])
dataset.shape
dataset[['latitude', 'longitude']].dropna().shape
from itertools import tee
def pairwise(iterable):
"s -> (s0,s1), (s1,s2), (s2, s3), ..."
a, b = tee(iterable)
next(b, None)
return zip(a, b)
def calculate_distances(x):
coordinate_list = x[['latitude', 'longitude']].values
distance_list = [distance(*coordinates_pair).km
for coordinates_pair in pairwise(coordinate_list)]
return np.nansum(distance_list)
distances = dataset.groupby(keys).apply(calculate_distances)
This way works, but the order of the visited cities is important. We need to find a way of calculate in the proper order or be fair in all calculations.
distances = distances.reset_index() \
.rename(columns={0: 'distance_traveled'}) \
.sort_values('distance_traveled', ascending=False)
distances.head()
Now we are not ordering the list of cities, just calculating the distance between them in the order they are in the dataset. Since we don't have the time of the expenses to know their real order, one approach is to consider the shortest path between in the cities visited in the day by the congressperson.
import networkx as nx
G = nx.Graph()
G=nx.path_graph(5)
G
path=nx.all_pairs_shortest_path(G)
path
path[0][4]
random_congressperson_day = cities[cities['city'] == 3].sample(random_state=0).reset_index().iloc[0]
matching_keys = ['congressperson_name', 'issue_date']
matches = \
(dataset['congressperson_name'] == random_congressperson_day['congressperson_name']) & \
(dataset['issue_date'] == random_congressperson_day['issue_date'])
expenses_for_graph = dataset[matches]
expenses_for_graph
def city_and_state(row):
return '{} - {}'.format(row['city'], row['state_y'])
expenses_for_graph['city_state'] = expenses_for_graph.apply(city_and_state, axis=1)
expenses_for_graph['city_state']
lat_longs = expenses_for_graph[['city_state', 'latitude', 'longitude']].values
# np.apply_along_axis(lambda x: (x[0], x[1]), axis=1, arr=lat_longs)
from itertools import combinations
list(combinations(lat_longs.tolist(), 2))
def create_node(row):
print(row[0], row[1], row[2])
cities_graph.add_node(row[0], pos=(row[1], row[2]))
return 42
cities_graph = nx.Graph()
np.apply_along_axis(create_node, axis=1, arr=lat_longs)
edges = list(combinations(lat_longs.tolist(), 2))
for edge in edges:
weight = distance(edge[0][1:], edge[1][1:]).km
print(edge[0][0], edge[1][0], weight)
cities_graph.add_edge(edge[0][0], edge[1][0], weight=weight)
# cities_graph.add_node('starting_point')
# new_edges = [('starting_point', node) for node in cities_graph.nodes()]
# cities_graph.add_edges_from(new_edges, weight=0)
cities_graph.nodes()
cities_graph.edges()
# Boworred from https://gist.github.com/mikkelam/ab7966e7ab1c441f947b
# Should we believe this algorithm is well implemented?
# Now is not the best way to learn how to do it ourselves...
def hamilton(G):
F = [(G,[G.nodes()[0]])]
n = G.number_of_nodes()
while F:
graph,path = F.pop()
confs = []
for node in graph.neighbors(path[-1]):
conf_p = path[:]
conf_p.append(node)
conf_g = nx.Graph(graph)
conf_g.remove_node(path[-1])
confs.append((conf_g,conf_p))
for g,p in confs:
if len(p)==n:
return p
else:
F.append((g,p))
return None
hamilton(cities_graph)
Calculating the minimum distance traveled each day is not so necessary, since we just need a number to say how far the visited cities are from each other. Summing the distances between all of their combinations is good enough.
def calculate_sum_distances(x):
coordinate_list = x[['latitude', 'longitude']].values
edges = list(combinations(coordinate_list, 2))
return np.sum([distance(edge[0][1:], edge[1][1:]).km for edge in edges])
distances = dataset.groupby(keys).apply(calculate_sum_distances)
distances = distances.reset_index() \
.rename(columns={0: 'distance_traveled'}) \
.sort_values('distance_traveled', ascending=False)
distances.head()
cities.reset_index()
aggregation = pd.merge(aggregation.reset_index(), cities.reset_index())
dataset_with_distances = \
pd.merge(aggregation,
distances,
left_on=keys,
right_on=keys)
dataset_with_distances.sort_values(['distance_traveled', 'expenses'], ascending=[False, False]).head(10)
%matplotlib inline
import seaborn as sns
sns.lmplot('expenses', 'distance_traveled',
data=dataset_with_distances,
scatter_kws={'marker': 'D',
's': 100},
size=10,
order=2)
dataset_with_distances.describe()
dataset_with_distances[dataset_with_distances['expenses'] > 4].shape
expenses_ceiling = dataset_with_distances['expenses'].mean() + \
(3 * dataset_with_distances['expenses'].std())
expenses_ceiling
distance_traveled_ceiling = dataset_with_distances['distance_traveled'].mean() + \
(3 * dataset_with_distances['distance_traveled'].std())
distance_traveled_ceiling
is_anomaly = (dataset_with_distances['expenses'] > expenses_ceiling) & \
(dataset_with_distances['distance_traveled'] > distance_traveled_ceiling)
dataset_with_distances[is_anomaly].shape
dataset_with_distances.loc[is_anomaly].sum()
dataset_with_distances.loc[is_anomaly, 'mean'].mean()
len(dataset_with_distances.loc[is_anomaly]) / len(dataset_with_distances)
dataset_with_distances[is_anomaly] \
.groupby('congressperson_name')['expenses'].count().reset_index() \
.rename(columns={'expenses': 'abnormal_days'}) \
.sort_values('abnormal_days', ascending=False) \
.head(10)
dataset_with_distances['3_stds_anomaly'] = is_anomaly
sns.lmplot('expenses', 'distance_traveled',
data=dataset_with_distances,
fit_reg=False,
hue='3_stds_anomaly',
scatter_kws={'marker': 'D',
's': 100},
size=10)
expenses_ceiling = dataset_with_distances['expenses'].mean() + \
(5 * dataset_with_distances['expenses'].std())
distance_traveled_ceiling = dataset_with_distances['distance_traveled'].mean() + \
(5 * dataset_with_distances['distance_traveled'].std())
is_anomaly = (dataset_with_distances['expenses'] > expenses_ceiling) & \
(dataset_with_distances['distance_traveled'] > distance_traveled_ceiling)
dataset_with_distances[is_anomaly].shape
expenses_ceiling, distance_traveled_ceiling
dataset_with_distances['5_stds_anomaly'] = is_anomaly
len(dataset_with_distances.loc[is_anomaly]) / len(dataset_with_distances)
sns.lmplot('expenses', 'distance_traveled',
data=dataset_with_distances,
fit_reg=False,
hue='5_stds_anomaly',
scatter_kws={'marker': 'D',
's': 100},
size=10)
dataset_with_distances[is_anomaly] \
.groupby('congressperson_name')['expenses'].count().reset_index() \
.rename(columns={'expenses': 'abnormal_days'}) \
.sort_values('abnormal_days', ascending=False) \
.head(10)
If we get similar results to this simple method, we expect to find the same congressperon, but not the same days. If we find the same days, our approach is not good enough compared to the simples method because we already that in the previous section we did not considered the combination of features but just their values compared to the distribuition of each feature (column). We expect the same congrespeople because the previous results shows that those person are travelling a lot more compared to the other congresspeople. For instance, Dr. Adilson Soares appears with 106 abnormal days more then twice than the second in the ranking, Sandra Rosado. Thus, would be expected to see him in the top ranking of congresspeople with abnormal meal expenses.
%matplotlib inline
import seaborn as sns
sns.set(color_codes=True)
sns.distplot(dataset_with_distances['expenses'],
bins=14,
kde=False)
sns.distplot(dataset_with_distances.query('1 < expenses < 8')['expenses'],
bins=6,
kde=False
)
query = '(1 < expenses < 8)'
sns.distplot(dataset_with_distances.query(query)['distance_traveled'],
bins=20,
kde=False)
query = '(1 < expenses < 8) & (0 < distance_traveled < 5000)'
sns.distplot(dataset_with_distances.query(query)['distance_traveled'],
bins=20,
kde=False)
from sklearn.ensemble import IsolationForest
predictor_keys = ['mean', 'expenses', 'sum', 'distance_traveled']
model = IsolationForest(random_state=0)
model.fit(dataset_with_distances[predictor_keys])
query = '(congressperson_name == "DR. ADILSON SOARES")'
expected_abnormal_day = dataset_with_distances[is_anomaly] \
.query(query) \
.sort_values('expenses', ascending=False).iloc[0]
expected_abnormal_day
model.predict([expected_abnormal_day[predictor_keys]])
y = model.predict(dataset_with_distances[predictor_keys])
dataset_with_distances['isolation_forest_anomaly'] = y == -1
dataset_with_distances['isolation_forest_anomaly'].sum()
Too many anomalies found.
sns.lmplot('expenses', 'distance_traveled',
data=dataset_with_distances,
fit_reg=False,
hue='isolation_forest_anomaly',
scatter_kws={'marker': 'D',
's': 100},
size=10)
How about changing the predictor keys for something more prone to trigger illegal expenses (# of expenses and distance between cities of expenses each day)?
predictor_keys = ['expenses', 'distance_traveled']
model = IsolationForest(contamination=.001, random_state=0)
model.fit(dataset_with_distances[predictor_keys])
model.predict([expected_abnormal_day[predictor_keys]])
y = model.predict(dataset_with_distances[predictor_keys])
dataset_with_distances['isolation_forest_anomaly'] = y == -1
dataset_with_distances['isolation_forest_anomaly'].sum()
sns.lmplot('expenses', 'distance_traveled',
data=dataset_with_distances,
fit_reg=False,
hue='isolation_forest_anomaly',
scatter_kws={'marker': 'D',
's': 100},
size=10)
dataset_with_distances.query('isolation_forest_anomaly') \
.groupby('congressperson_name')['expenses'].count().reset_index() \
.rename(columns={'expenses': 'abnormal_days'}) \
.sort_values('abnormal_days', ascending=False) \
.head(10)
# Authors: Nicolas Goix<EMAIL_ADDRESS># Alexandre Gramfort<EMAIL_ADDRESS># License: BSD 3 clause
import numpy as np
from warnings import warn
from scipy.stats import scoreatpercentile
from sklearn.neighbors.base import NeighborsBase
from sklearn.neighbors.base import KNeighborsMixin
from sklearn.neighbors.base import UnsupervisedMixin
from sklearn.utils.validation import check_is_fitted
from sklearn.utils import check_array
__all__ = ["LocalOutlierFactor"]
class LocalOutlierFactor(NeighborsBase, KNeighborsMixin, UnsupervisedMixin):
"""Unsupervised Outlier Detection using Local Outlier Factor (LOF)
The anomaly score of each sample is called Local Outlier Factor.
It measures the local deviation of density of a given sample with
respect to its neighbors.
It is local in that the anomaly score depends on how isolated the object
is with respect to the surrounding neighborhood.
More precisely, locality is given by k-nearest neighbors, whose distance
is used to estimate the local density.
By comparing the local density of a sample to the local densities of
its neighbors, one can identify samples that have a substantially lower
density than their neighbors. These are considered outliers.
Parameters
----------
n_neighbors : int, optional (default=20)
Number of neighbors to use by default for :meth:`kneighbors` queries.
If n_neighbors is larger than the number of samples provided,
all samples will be used.
algorithm : {'auto', 'ball_tree', 'kd_tree', 'brute'}, optional
Algorithm used to compute the nearest neighbors:
- 'ball_tree' will use :class:`BallTree`
- 'kd_tree' will use :class:`KDTree`
- 'brute' will use a brute-force search.
- 'auto' will attempt to decide the most appropriate algorithm
based on the values passed to :meth:`fit` method.
Note: fitting on sparse input will override the setting of
this parameter, using brute force.
leaf_size : int, optional (default=30)
Leaf size passed to :class:`BallTree` or :class:`KDTree`. This can
affect the speed of the construction and query, as well as the memory
required to store the tree. The optimal value depends on the
nature of the problem.
p : integer, optional (default=2)
Parameter for the Minkowski metric from
:ref:`sklearn.metrics.pairwise.pairwise_distances`. When p = 1, this is
equivalent to using manhattan_distance (l1), and euclidean_distance
(l2) for p = 2. For arbitrary p, minkowski_distance (l_p) is used.
metric : string or callable, default 'minkowski'
metric used for the distance computation. Any metric from scikit-learn
or scipy.spatial.distance can be used.
If 'precomputed', the training input X is expected to be a distance
matrix.
If metric is a callable function, it is called on each
pair of instances (rows) and the resulting value recorded. The callable
should take two arrays as input and return one value indicating the
distance between them. This works for Scipy's metrics, but is less
efficient than passing the metric name as a string.
Valid values for metric are:
- from scikit-learn: ['cityblock', 'cosine', 'euclidean', 'l1', 'l2',
'manhattan']
- from scipy.spatial.distance: ['braycurtis', 'canberra', 'chebyshev',
'correlation', 'dice', 'hamming', 'jaccard', 'kulsinski',
'mahalanobis', 'matching', 'minkowski', 'rogerstanimoto',
'russellrao', 'seuclidean', 'sokalmichener', 'sokalsneath',
'sqeuclidean', 'yule']
See the documentation for scipy.spatial.distance for details on these
metrics:
http://docs.scipy.org/doc/scipy/reference/spatial.distance.html
metric_params : dict, optional (default=None)
Additional keyword arguments for the metric function.
contamination : float in (0., 0.5), optional (default=0.1)
The amount of contamination of the data set, i.e. the proportion
of outliers in the data set. When fitting this is used to define the
threshold on the decision function.
n_jobs : int, optional (default=1)
The number of parallel jobs to run for neighbors search.
If ``-1``, then the number of jobs is set to the number of CPU cores.
Affects only :meth:`kneighbors` and :meth:`kneighbors_graph` methods.
Attributes
----------
negative_outlier_factor_ : numpy array, shape (n_samples,)
The opposite LOF of the training samples. The lower, the more normal.
Inliers tend to have a LOF score close to 1, while outliers tend
to have a larger LOF score.
The local outlier factor (LOF) of a sample captures its
supposed 'degree of abnormality'.
It is the average of the ratio of the local reachability density of
a sample and those of its k-nearest neighbors.
n_neighbors_ : integer
The actual number of neighbors used for :meth:`kneighbors` queries.
References
----------
.. [1] Breunig, M. M., Kriegel, H. P., Ng, R. T., & Sander, J. (2000, May).
LOF: identifying density-based local outliers. In ACM sigmod record.
"""
def __init__(self, n_neighbors=20, algorithm='auto', leaf_size=30,
metric='minkowski', p=2, metric_params=None,
contamination=0.1, n_jobs=1):
self._init_params(n_neighbors=n_neighbors,
algorithm=algorithm,
leaf_size=leaf_size, metric=metric, p=p,
metric_params=metric_params, n_jobs=n_jobs)
self.contamination = contamination
def fit_predict(self, X, y=None):
""""Fits the model to the training set X and returns the labels
(1 inlier, -1 outlier) on the training set according to the LOF score
and the contamination parameter.
Parameters
----------
X : array-like, shape (n_samples, n_features), default=None
The query sample or samples to compute the Local Outlier Factor
w.r.t. to the training samples.
Returns
-------
is_inlier : array, shape (n_samples,)
Returns 1 for anomalies/outliers and -1 for inliers.
"""
return self.fit(X)._predict()
def fit(self, X, y=None):
"""Fit the model using X as training data.
Parameters
----------
X : {array-like, sparse matrix, BallTree, KDTree}
Training data. If array or matrix, shape [n_samples, n_features],
or [n_samples, n_samples] if metric='precomputed'.
Returns
-------
self : object
Returns self.
"""
if not (0. < self.contamination <= .5):
raise ValueError("contamination must be in (0, 0.5]")
super(LocalOutlierFactor, self).fit(X)
n_samples = self._fit_X.shape[0]
if self.n_neighbors > n_samples:
warn("n_neighbors (%s) is greater than the "
"total number of samples (%s). n_neighbors "
"will be set to (n_samples - 1) for estimation."
% (self.n_neighbors, n_samples))
self.n_neighbors_ = max(1, min(self.n_neighbors, n_samples - 1))
self._distances_fit_X_, _neighbors_indices_fit_X_ = (
self.kneighbors(None, n_neighbors=self.n_neighbors_))
self._lrd = self._local_reachability_density(
self._distances_fit_X_, _neighbors_indices_fit_X_)
# Compute lof score over training samples to define threshold_:
lrd_ratios_array = (self._lrd[_neighbors_indices_fit_X_] /
self._lrd[:, np.newaxis])
self.negative_outlier_factor_ = -np.mean(lrd_ratios_array, axis=1)
self.threshold_ = -scoreatpercentile(
-self.negative_outlier_factor_, 100. * (1. - self.contamination))
return self
def _predict(self, X=None):
"""Predict the labels (1 inlier, -1 outlier) of X according to LOF.
If X is None, returns the same as fit_predict(X_train).
This method allows to generalize prediction to new observations (not
in the training set). As LOF originally does not deal with new data,
this method is kept private.
Parameters
----------
X : array-like, shape (n_samples, n_features), default=None
The query sample or samples to compute the Local Outlier Factor
w.r.t. to the training samples. If None, makes prediction on the
training data without considering them as their own neighbors.
Returns
-------
is_inlier : array, shape (n_samples,)
Returns -1 for anomalies/outliers and +1 for inliers.
"""
check_is_fitted(self, ["threshold_", "negative_outlier_factor_",
"n_neighbors_", "_distances_fit_X_"])
if X is not None:
X = check_array(X, accept_sparse='csr')
is_inlier = np.ones(X.shape[0], dtype=int)
is_inlier[self._decision_function(X) <= self.threshold_] = -1
else:
is_inlier = np.ones(self._fit_X.shape[0], dtype=int)
is_inlier[self.negative_outlier_factor_ <= self.threshold_] = -1
return is_inlier
def _decision_function(self, X):
"""Opposite of the Local Outlier Factor of X (as bigger is better,
i.e. large values correspond to inliers).
The argument X is supposed to contain *new data*: if X contains a
point from training, it consider the later in its own neighborhood.
Also, the samples in X are not considered in the neighborhood of any
point.
The decision function on training data is available by considering the
opposite of the negative_outlier_factor_ attribute.
Parameters
----------
X : array-like, shape (n_samples, n_features)
The query sample or samples to compute the Local Outlier Factor
w.r.t. the training samples.
Returns
-------
opposite_lof_scores : array, shape (n_samples,)
The opposite of the Local Outlier Factor of each input samples.
The lower, the more abnormal.
"""
check_is_fitted(self, ["threshold_", "negative_outlier_factor_",
"_distances_fit_X_"])
X = check_array(X, accept_sparse='csr')
distances_X, neighbors_indices_X = (
self.kneighbors(X, n_neighbors=self.n_neighbors_))
X_lrd = self._local_reachability_density(distances_X,
neighbors_indices_X)
lrd_ratios_array = (self._lrd[neighbors_indices_X] /
X_lrd[:, np.newaxis])
# as bigger is better:
return -np.mean(lrd_ratios_array, axis=1)
def _local_reachability_density(self, distances_X, neighbors_indices):
"""The local reachability density (LRD)
The LRD of a sample is the inverse of the average reachability
distance of its k-nearest neighbors.
Parameters
----------
distances_X : array, shape (n_query, self.n_neighbors)
Distances to the neighbors (in the training samples `self._fit_X`)
of each query point to compute the LRD.
neighbors_indices : array, shape (n_query, self.n_neighbors)
Neighbors indices (of each query point) among training samples
self._fit_X.
Returns
-------
local_reachability_density : array, shape (n_samples,)
The local reachability density of each sample.
"""
dist_k = self._distances_fit_X_[neighbors_indices,
self.n_neighbors_ - 1]
reach_dist_array = np.maximum(distances_X, dist_k)
# 1e-10 to avoid `nan' when when nb of duplicates > n_neighbors_:
return 1. / (np.mean(reach_dist_array, axis=1) + 1e-10)
# predictor_keys = ['mean', 'expenses', 'sum', 'distance_traveled']
predictor_keys = ['expenses', 'distance_traveled']
model = LocalOutlierFactor(n_jobs=-1)
y = model.fit_predict(dataset_with_distances[predictor_keys])
model._predict([expected_abnormal_day[predictor_keys]])
dataset_with_distances['lof_anomaly'] = y == -1
dataset_with_distances['lof_anomaly'].sum()
sns.lmplot('expenses', 'distance_traveled',
data=dataset_with_distances,
fit_reg=False,
hue='lof_anomaly',
scatter_kws={'marker': 'D',
's': 100},
size=10)
Let's see the results using just distance_traveled
as predictor.
predictor_keys = ['distance_traveled']
model = LocalOutlierFactor(contamination=.01, n_jobs=-1)
y = model.fit_predict(dataset_with_distances[predictor_keys])
model._predict([expected_abnormal_day[predictor_keys]])
dataset_with_distances['lof_anomaly'] = y == -1
dataset_with_distances['lof_anomaly'].sum()
sns.lmplot('expenses', 'distance_traveled',
data=dataset_with_distances.query('lof_anomaly'),
fit_reg=False,
hue='lof_anomaly',
scatter_kws={'marker': 'D',
's': 100},
size=10)
The congresspeople ranking is similar to the one using standard deviation method.
dataset_with_distances.query('lof_anomaly') \
.groupby('congressperson_name')['expenses'].count().reset_index() \
.rename(columns={'expenses': 'abnormal_days'}) \
.sort_values('abnormal_days', ascending=False) \
.head(10)
How about trying to combine standard deviation and Local Outlier Factor?
to_show = dataset_with_distances['lof_anomaly'] & \
dataset_with_distances['3_stds_anomaly']
sns.lmplot('expenses', 'distance_traveled',
data=dataset_with_distances[to_show],
fit_reg=False,
hue='lof_anomaly',
scatter_kws={'marker': 'D',
's': 100},
size=10)
dataset_with_distances[to_show].shape
dataset_with_distances[to_show]
dataset_with_distances['lof_anomaly'].describe()
Number of cities may not be reflected in distance_traveled
already. Let's see the differencies in the results.
predictor_keys = ['distance_traveled', 'city']
model = LocalOutlierFactor(contamination=.01, n_jobs=-1)
y = model.fit_predict(dataset_with_distances[predictor_keys])
model._predict([expected_abnormal_day[predictor_keys]])
dataset_with_distances['lof_anomaly'] = y == -1
dataset_with_distances['lof_anomaly'].sum()
dataset_with_distances.query('lof_anomaly') \
.groupby('congressperson_name')['expenses'].count().reset_index() \
.rename(columns={'expenses': 'abnormal_days'}) \
.sort_values('abnormal_days', ascending=False) \
.head(10)
to_show = dataset_with_distances['lof_anomaly']
sns.lmplot('expenses', 'distance_traveled',
data=dataset_with_distances[to_show],
fit_reg=False,
hue='lof_anomaly',
scatter_kws={'marker': 'D',
's': 100},
size=10)
query = '(lof_anomaly == True) & (congressperson_name == "VANDERLEI MACRIS")'
dataset_with_distances.query(query).sort_values('issue_date', ascending=False).head()
So far, IsolationForest
using expenses
and distance_traveled
as predictors seem to have best results for the purpose of this analysis: allowing deputies to make many expenses (even expensive ones) and traveling through many cities in a single day, but questioning those making too many expenses without the excuse of a business trip.
5-stds and IsolationForest
with expenses/distance_traveled
as predictors and contamination of 0.1% return the best results. We need to compare them before going further.
predictor_keys = ['expenses', 'distance_traveled']
forest_model = IsolationForest(contamination=.001, random_state=0)
forest_model.fit(dataset_with_distances[predictor_keys])
dataset_with_distances.query('distance_traveled == {}'.format(dataset_with_distances['distance_traveled'].max()))
isolation_ranking = dataset_with_distances.query('isolation_forest_anomaly') \
.groupby('congressperson_name')['expenses'].count().reset_index() \
.rename(columns={'expenses': 'abnormal_days'}) \
.sort_values('abnormal_days', ascending=False)
forest_anomaly_congresspeople = set(isolation_ranking['congressperson_name'])
len(forest_anomaly_congresspeople)
stds_ranking = dataset_with_distances[dataset_with_distances['5_stds_anomaly']] \
.groupby('congressperson_name')['expenses'].count().reset_index() \
.rename(columns={'expenses': 'abnormal_days'}) \
.sort_values('abnormal_days', ascending=False)
stds_anomaly_congresspeople = set(stds_ranking['congressperson_name'])
len(stds_anomaly_congresspeople)
Congresspeople who are both in IsolationForest
and 5-stds
models:
common_congresspeople = forest_anomaly_congresspeople & stds_anomaly_congresspeople
common_congresspeople
Congresspeople who are just in IsolationForest
or 5-stds
model:
forest_anomaly_congresspeople ^ stds_anomaly_congresspeople
congresspeople_ranking = pd.merge(isolation_ranking, stds_ranking,
how='outer',
on='congressperson_name',
suffixes=('_isolation', '_stds'))
congresspeople_ranking['isolation_stds_diff'] = \
abs(congresspeople_ranking['abnormal_days_isolation'] - \
congresspeople_ranking['abnormal_days_stds'])
congresspeople_ranking.head()
congresspeople_ranking['ranking'] = \
np.mean(congresspeople_ranking[['abnormal_days_isolation', 'abnormal_days_stds']], axis=1) - congresspeople_ranking['isolation_stds_diff']
congresspeople_ranking.sort_values('ranking', ascending=False)
is_in_both_models = \
dataset_with_distances['congressperson_name'].isin(common_congresspeople) & \
dataset_with_distances['5_stds_anomaly'] & \
dataset_with_distances['isolation_forest_anomaly']
dataset_with_distances[is_in_both_models].sort_values('distance_traveled', ascending=False)
dataset_with_distances['lof_and_5_stds'] = dataset_with_distances['isolation_forest_anomaly'] & dataset_with_distances['5_stds_anomaly']
dataset_with_distances['lof_or_5_stds'] = dataset_with_distances['isolation_forest_anomaly'] | dataset_with_distances['5_stds_anomaly']
sns.lmplot('expenses', 'distance_traveled',
data=dataset_with_distances,
fit_reg=False,
hue='5_stds_anomaly',
scatter_kws={'marker': 'D',
's': 100},
size=10)
sns.lmplot('expenses', 'distance_traveled',
data=dataset_with_distances,
fit_reg=False,
hue='isolation_forest_anomaly',
scatter_kws={'marker': 'D',
's': 100},
size=10)
sns.lmplot('expenses', 'distance_traveled',
data=dataset_with_distances,
fit_reg=False,
hue='lof_and_5_stds',
scatter_kws={'marker': 'D',
's': 100},
size=10)
sns.lmplot('expenses', 'distance_traveled',
data=dataset_with_distances,
fit_reg=False,
hue='lof_or_5_stds',
scatter_kws={'marker': 'D',
's': 100},
size=10)
dataset['issue_date_day'] = dataset['issue_date'].apply(lambda date: date.day)
dataset['issue_date_month'] = dataset['issue_date'].apply(lambda date: date.month)
dataset['issue_date_year'] = dataset['issue_date'].apply(lambda date: date.year)
dataset['issue_date_weekday'] = dataset['issue_date'].apply(lambda date: date.weekday())
dataset['issue_date_week'] = dataset['issue_date'].apply(lambda date: date.week)
# keys = ['applicant_id', 'issue_date_week', 'issue_date_year']
keys = ['congressperson_name', 'issue_date_week', 'issue_date_year']
aggregation_weeks = dataset.groupby(keys)['total_net_value']. \
agg({'sum': np.sum, 'expenses': len, 'mean': np.mean})
aggregation_weeks['expenses'] = aggregation_weeks['expenses'].astype(np.int)
aggregation_weeks.sort_values(['expenses', 'sum'], ascending=[False, False]).head(10)
cities_weeks = dataset.groupby(keys)['city']. \
agg({'city': lambda x: len(set(x)),
'city_list': lambda x: ','.join(set(x))}
).sort_values('city', ascending=False)
cities_weeks.head()
def calculate_sum_distances(x):
coordinate_list = x[['latitude', 'longitude']].values
edges = list(combinations(coordinate_list, 2))
return np.sum([distance(edge[0][1:], edge[1][1:]).km for edge in edges])
distances_weeks = dataset.groupby(keys).apply(calculate_sum_distances)
distances_weeks = distances_weeks.reset_index() \
.rename(columns={0: 'distance_traveled'}) \
.sort_values('distance_traveled', ascending=False)
distances_weeks.head()
aggregation_weeks = pd.merge(aggregation_weeks.reset_index(), cities_weeks.reset_index())
dataset_with_distances_weeks = \
pd.merge(aggregation_weeks,
distances_weeks,
left_on=keys,
right_on=keys)
dataset_with_distances_weeks.sort_values(['distance_traveled', 'expenses'], ascending=[False, False]).head(10)
sns.lmplot('expenses', 'distance_traveled',
data=dataset_with_distances_weeks,
scatter_kws={'marker': 'D',
's': 100},
size=10,
order=3)
dataset_with_distances_weeks.describe()
4 "meals" every weekday
dataset_with_distances_weeks[dataset_with_distances_weeks['expenses'] > 20].shape
expenses_ceiling = dataset_with_distances_weeks['expenses'].mean() + \
(5 * dataset_with_distances_weeks['expenses'].std())
distance_traveled_ceiling = dataset_with_distances_weeks['distance_traveled'].mean() + \
(5 * dataset_with_distances_weeks['distance_traveled'].std())
is_anomaly = (dataset_with_distances_weeks['expenses'] > expenses_ceiling) & \
(dataset_with_distances_weeks['distance_traveled'] > distance_traveled_ceiling)
dataset_with_distances_weeks[is_anomaly].shape
expenses_ceiling, distance_traveled_ceiling
dataset_with_distances_weeks[is_anomaly] \
.groupby('congressperson_name')['expenses'].count().reset_index() \
.rename(columns={'expenses': 'abnormal_weeks'}) \
.sort_values('abnormal_weeks', ascending=False) \
.head(10)
dataset_with_distances_weeks['5_stds_anomaly'] = is_anomaly
dataset_with_distances_weeks['5_stds_anomaly'].sum() / len(dataset_with_distances_weeks)
sns.lmplot('expenses', 'distance_traveled',
data=dataset_with_distances_weeks,
fit_reg=False,
hue='5_stds_anomaly',
scatter_kws={'marker': 'D',
's': 100},
size=10)
predictor_keys = ['issue_date_week', 'issue_date_year', 'expenses', 'distance_traveled']
model = IsolationForest(contamination=.001, random_state=0)
model.fit(dataset_with_distances_weeks[predictor_keys])
y = model.predict(dataset_with_distances_weeks[predictor_keys])
dataset_with_distances_weeks['isolation_forest_anomaly'] = y == -1
dataset_with_distances_weeks['isolation_forest_anomaly'].sum()
sns.lmplot('expenses', 'distance_traveled',
data=dataset_with_distances_weeks,
fit_reg=False,
hue='isolation_forest_anomaly',
scatter_kws={'marker': 'D',
's': 100},
size=10)
Ideas
expenses_threshold = dataset_with_distances['expenses'].mean() + \
(10 * dataset_with_distances['expenses'].std())
dataset_with_distances['my_anomaly'] = dataset_with_distances['expenses'] > 8
dataset_with_distances['my_and_forest_anomaly'] = \
dataset_with_distances['my_anomaly'] | \
dataset_with_distances['isolation_forest_anomaly']
print(dataset_with_distances['my_anomaly'].sum())
sns.lmplot('expenses', 'distance_traveled',
data=dataset_with_distances,
hue='my_anomaly',
scatter_kws={'marker': 'D',
's': 100},
size=10,
fit_reg=False)
dataset_with_distances[dataset_with_distances['my_anomaly']] \
.groupby('congressperson_name')['expenses'].count().reset_index() \
.rename(columns={'expenses': 'abnormal_days'}) \
.sort_values('abnormal_days', ascending=False) \
.head(10)
In fact, getting feedback from other people from the team, I came to the conclusion that the cases we want to report as "anomalies" are related with a regression.
First: cases with more than 8 meal receipts in a single day we want to get reported, doesn't matter the distance. Secondly: people traveling are expected to request for more meal reimbursements (e.g. coffee before going to the airport, something quick before checkin, more calm breakfast waiting for the flight... all of this in the same day, still legal), but many receipts in the same day, with short distance traveled is suspect. Still, too many receipts is still suspect. Thus what we expect is to form a regression, give an acceptable margin and consider everything outside, above and below it, suspect.
to_plot = pd.DataFrame()
to_plot['x'] = dataset_with_distances['expenses'].astype(np.long)
fit = np.polyfit(to_plot['x'],
dataset_with_distances['distance_traveled'].astype(np.long),
3)
print(fit)
fit_fn = np.poly1d(fit)
to_plot['y'] = fit_fn(dataset_with_distances['expenses'])
sns.lmplot('x', 'y',
data=to_plot,
scatter_kws={'marker': 'D',
's': 100},
fit_reg=False,
size=10)
dataset_with_distances['expected_distance'] = \
fit_fn(dataset_with_distances['expenses'])
We want to limit the contamination to 0.1% (diff_distance
approximately 3,000).
expected_distances = dataset_with_distances['expected_distance']
diff_distance = abs(expected_distances - dataset_with_distances['distance_traveled'])
def contamination(threshold):
return (diff_distance > threshold).sum() / \
(len(dataset_with_distances) - dataset_with_distances['my_anomaly'].sum())
def threshold_for_contamination(expected_contamination):
possible_thresholds = range(1, int(expected_distances.max()), 50)
results = [(contamination(x), x) for x in possible_thresholds]
best_choice = min(results, key=lambda x: abs(x[0] - expected_contamination))
return best_choice[1]
threshold_for_contamination(.001)
dataset_with_distances['polyfit_anomaly'] = \
diff_distance > threshold_for_contamination(.001)
dataset_with_distances['polyfit_anomaly'].sum()
dataset_with_distances['my_and_polyfit_anomaly'] = \
dataset_with_distances['my_anomaly'] | \
dataset_with_distances['polyfit_anomaly']
dataset_with_distances['my_and_polyfit_anomaly'].sum()
sns.lmplot('expenses', 'distance_traveled',
data=dataset_with_distances,
hue='my_and_polyfit_anomaly',
scatter_kws={'marker': 'D',
's': 100},
fit_reg=False,
size=10)
dataset_with_distances.query('my_and_polyfit_anomaly & (expenses == 3)')
dataset.query('(congressperson_name == "ROCHA") & (issue_date == "2015-06-19")')['document_id'].values
dataset_with_distances.query('my_and_polyfit_anomaly') \
.groupby('congressperson_name')['expenses'].count().reset_index() \
.rename(columns={'expenses': 'abnormal_days'}) \
.sort_values('abnormal_days', ascending=False) \
.head(10)
dataset_with_distances.query('my_and_polyfit_anomaly')['expenses'].sum()
dataset_with_distances.query('my_and_polyfit_anomaly')['sum'].sum()
The results from "polyfit anomaly" don't necessarily mean anomalies, neither the final approach can really be consider a Anomaly Detection method. Yet is able to alert about a few cases deserving auditing and further investigation to prove illegal or immoral activities:
Ana Schwendler was my pair during a good portion of the time working on this analysis. Felipe Cabral also gave a nice guidance helping to define anomalous expenses and brainstorming approaches.


Interface IAsyncObserver<T>
This interface generalizes the standard .NET IObserver interface to allow asynchronous production of items.
Note that this interface is implemented by item consumers and invoked (used) by item producers. This means that the consumer endpoint of a stream implements this interface.
Namespace: Orleans.Streams
Assembly: Orleans.Core.Abstractions.dll
Syntax
public interface IAsyncObserver<in T>
Type Parameters
| Name | Description |
|---|---|
| T | The type of object consumed by the observer. |
Methods
| Improve this Doc View SourceOnCompletedAsync()
Notifies the consumer that the stream was completed.
The Task returned from this method should be completed when the consumer is done processing the stream closure.
Declaration
Task OnCompletedAsync()
Returns
| Type | Description |
|---|---|
| Task | A Task that is completed when the stream-complete operation has been accepted. |
OnErrorAsync(Exception)
Notifies the consumer that the stream had an error.
The Task returned from this method should be completed when the consumer is done processing the stream closure.
Declaration
Task OnErrorAsync(Exception ex)
Parameters
| Type | Name | Description |
|---|---|---|
| Exception | ex | An Exception that describes the error that occured on the stream. |
Returns
| Type | Description |
|---|---|
| Task | A Task that is completed when the close has been accepted. |
OnNextAsync(T, StreamSequenceToken)
Passes the next item to the consumer.
The Task returned from this method should be completed when the item's processing has been sufficiently processed by the consumer to meet any behavioral guarantees.
When the consumer is the (producer endpoint of) a stream, the Task is completed when the stream implementation has accepted responsibility for the item and is assured of meeting its delivery guarantees. For instance, a stream based on a durable queue would complete the Task when the item has been durably saved. A stream that provides best-effort at most once delivery would return a Task that is already complete.
When the producer is the (consumer endpoint of) a stream, the Task should be completed by the consumer code when it has accepted responsibility for the item. In particular, if the stream provider guarantees at-least-once delivery, then the item should not be considered delivered until the Task returned by the consumer has been completed.
Declaration
Task OnNextAsync(T item, StreamSequenceToken token = null)
Parameters
| Type | Name | Description |
|---|---|---|
| T | item | The item to be passed. |
| StreamSequenceToken | token | The stream sequence token of this item. |
Returns
| Type | Description |
|---|---|
| Task | A Task that is completed when the item has been accepted. |


Texas Southern beats University of Hawaii Wahine basketball team | Honolulu Star-Advertiser
Richelle Velez scored 25 points and Niya Mitchell 24 as visiting Texas Southern beat Hawaii 74-57 in a women’s college basketball game Sunday at the Stan Sheriff Center.
Amy Atwell led the Rainbow Wahine with 15 points.
Hawaii fell to 3-2. Texas Southern improved to 2-3.


Patient: Vivek Dwivedi
Patient Statistics -
100.2
Avg. Temperature
8
Days since surgery
6.5
Pain Level
Heart Rate
Total Visits
4


Alone on Nature
Here you can place an optional description of your Project
The Other Side of Me
Here you can place an optional description of your Project
Forever Young
Here you can place an optional description of your Project
Travelling is Fun
Here you can place an optional description of your Project
Norway Nature
Here you can place an optional description of your Project
A Wonderful Life
Here you can place an optional description of your Project


You need to enable JavaScript to run this app.
Boots Events Manager
Pending Event
Title
Description
Start Date
End Date
Actions


E-Tutor
Log Out
Dashboard
Group
View Group
Add New Group
Comments
Users
All Users
Add New User
Allocate Students to Tutors
Meeting
View Meeting
Add New Meeting
Issue
Profile
×
Choose a Tutor
×
Choose the Students
×
Profile of User
×
Enter Group Information
Enter Group Name:
Next
Cancel
×
Select Group Members
Add New Meeting
Title
Description
Date
Select Participants
×
Select Participants
Add New Meeting


Steven Koenig is studying the production of microbial exopolysaccharides using renewable resources at Technische Universität München, Straubing Center of Science, Chair of Chemistry of Biogenic Resources as a Ph.D. candidate.


{{ Many OS and Hardware manufacturers are making updates to help improve the transition to IPv6. Some known "gotchas" are being addressed by workarounds by these vendors. }}
{{ You can do your part to make sure you are running the latest: }}
{{ OS - Make sure you have the latest. }}
{{ Mac users: Apple OS X 10.6.7 is recommended for improved IPv6 support. }}
{{ Mac users on 10.4 or 10.5 may need to disable IPv6 if other troubleshooting steps fail. }}
{{ iPhone/iPad users: No specific version advised at this time. }}
{{ Routers - Make sure you have the latest firmware installed. Some vendors have updated the defaults, or put in health checks before enabling IPv6. }}
{{ If you are using an Apple router or Time Capsule, load the Airport Utility to check that your router is up to date. }}
{{ Browsers - Some browsers have made updates to detect "broken" users, and to temporarily disable IPv6. }}
{{ Firefox 4.x now only tries IPv6, when an IPv6 address is configured. Firefox 3 users are encouraged to upgrade. }}
{{ Chrome: If a wrench icon appears on the browser toolbar, click it. Select "Update Google Chrome" then Restart. }}


The LarKC Data Layer API describes the different types of RDF data providers and their functionality.


A collection of dynamically generated color
and background-color
utility classes.
JigSass Utils Color works in tandem with JigSass Tools Color
and allows defining color and background-color classes based on the color palette defined in
$jigsass-colors as well as
adjusting them on the fly.
Color utils can be limited to a certain state (i.e., hover
, focus
, etc.)
The syntax for generating a color util is: $modifier: '<color-name>[(<adjustment-function>,<arg>)][:state]'
Class names follow the Emmet abbreviation
syntax, with colons (':') replaced by two dashes (--
) to follow BEM naming
conventions.
Available classes
u-c--<color-name>
(example: u-c--brand)
u-c--<color-name>(<adjustment-func>,<arg>)
(example: u-c--brand(tint-2))
u-c--<color-name>:<state>
(example: u-c--brand:focus)
u-c--<color-name>(<adjustment-func>,<arg>):<state>
(example: u-c--brand(shade-20%):focus)
u-bgc--<color-name>
u-bgc--<color-name>(<adjustment-func>,<arg>)
u-bgc--<color-name>:<state>
u-bgc--<color-name>(<adjustment-func>,<arg>):<state>
Installation
Using npm:
npm i -S jigsass-utils-color
Usage
Import JigSass Utils Color into your main scss file near its very end, together with all
other utilities (utilities should always be the last to be imported).
@import 'path/to/jigsass-utils-color/scss/index';
Like all other JigSass Utils, JigSass Utils Color does not automatically generate any CSS
when imported. You would need to explicitly indicate that each individual color
class should actually be generated in each component or object it is used in
(clarification: This will include style declarations inside .foo
and .bar
):
// _c.foo.scss
.foo {
@include jigsass-util(u-c, $modifier: primary); // <-- color: #09a5d9
...
}
// _c.bar.scss
.bar {
@include jigsass-util(u-c, $modifier: 'black(tint,10%)'); // <-- color: #191919
@include jigsass-util(u-bgc, $modifier: 'white(shade,30%)', $from: large); // <-- background-color: #b2b2b2 from large bp and on.
...
}
Doing so helps us a great deal with portability, as no matter where we import component or object
partials, the correct utility classes will be generated. Think of it as a poor man's dependency
management.
Developer communication is also assisted by including "dependencies" wherever they are required,
as anyone going through a partial, can easily understand how it should be marked up with just a
glance.
As far as bloat goes, just don't worry about it - the actual styles will only be generated once,
at the location in the cascade where the Jigsass Utils Color partial was imported into the main file.
JigSass Utils Color classes are responsive-enabled, using JigSass MQ
and the breakpoints defined in the $jigsass-breakpoints variable.
Based on the breakpoint arguments passed to jigsass-util
when including a class,
responsive modifiers are generated according to the following logic:
.u-c-<modifier>[-[-from-<breakpoint-name>][-until-<breakpoint-name>][-misc-<breakpoint-name>]]
.u-bgc-<modifier>[-[-from-<breakpoint-name>][-until-<breakpoint-name>][-misc-<breakpoint-name>]]
So, assuming the medium
, large
and landscape
breakpoints are defined in $jigsass-breakpoints
as 600px
, 1024px
and (orientation: landscape)
respectively,
@include jigsass-util(u-c, $modifier: secondary);
will generate the .u-c--secondary
class, which is not limited to any media-query.
@include jigsass-util(u-bgc, $modifier: primary, $until: medium);
will generate the .u-bgc--primary--until-medium
class, which will be in effect at
(max-width: 37.49em)
and will override styles in the default class until that point.
@include jigsass-util(u-c, $modifier: primary, $from: large, $misc: landscape);
will generate the .u-c--primary--from-large-when-landscape
class, which will go into
effect at (min-width: 64em) and (orientation: landscape)
and will override styles in the default
class under these conditions.
License: MIT


In 1906, Emma Goldman started a journal. Known as “Mother Earth”, this journal would be a place of expression for anarchists and idealists of all types. During its history, Mother Earth would publish articles on a broad variety of topics, such as the labor movement, education, literature, government control, women’s emancipation, and sexual freedom.
In 1915, Mother Earth became one of the earliest supporters of birth control, bringing Goldman into conflict with the Comstock law, which prohibited the dissemination of “obscene” materials, including birth control information.
In 1917, Mother Earth openly called for its readers to disobey the laws on conscription and the military draft, as Goldman considered any war waged by capitalist governments to be illegitimate. As a result, Emma Goldman was tried under the Espionage Act, sentenced to two years in jail, and later deported. Mother Earth ceased publication.
Throughout the history of the magazine, Goldman would continually champion the causes of social justice and equality, in all their forms. She never wavered, even in the face of persistent government persecution.
What she did NOT do, however, was publish a wagonload of cliché radical pamphlets, whose pages read like haphazard excerpts from Baby’s First Radical Textbook. Look at this: “The university cannot be an ivory tower, far away from the society, removed from the practical accomplishments of the Revolution. Democracy cannot consist solely of elections that are nearly always fictitious and managed by rich landowners and professional politicians. We should not allow the word 'democracy' to be utilised apologetically to represent the dictatorship of the exploiting classes.”
To call this bad writing would be an insult to the concept of writing. This is not writing at all. It’s stringing together clichés, like lights on a Christmas tree.
Compare this insipid nonsense to Emma Goldman in the first issue of Mother Earth: “Mother Earth will endeavor to attract and appeal to all those who oppose encroachment on public and individual life. It will appeal to those who strive for something higher, weary of the commonplace; to those who feel that stagnation is a deadweight on the firm and elastic step of progress; to those who breathe freely only in limitless space; to those who long for the tender shade of a new dawn for a humanity free from the dread of want, the dread of starvation in the face of mountains of riches. The Earth free for the free individual!”
Now, that’s inspirational! Don’t just repeat what you’ve read. Write from the heart.
Bizarrely, the pamphlet also contains a quote from Philip Larkin, on the difficulty of forming informed political opinions. One can only hope that this shows some degree of budding self-awareness in the author(s). With luck, he/she/they might stop writing pamphlets, and read a book instead.
All this said, the pamphlet does seem to be advocating the use of violence in the pursuit of revolutionary goals. It’s hard to tell how much of this is the earnest opinion of the author(s), and how much is merely regurgitated from various left-wing sources, but it’s probably best not to take chances.
Recommendation: Grade B surveillance.