Haskell Generics are a somewhat misunderstood topic but are an extremely powerful technique for writing reusable and comparable interfaces across an enormous universe of types with very little effort. They are probably my favorite example of the advantages of an expressive type system endows us with.
Source for examples code is available here.
Generics are a form of datatype-generic programming, which although the namesake has some similarity to Java Generics they are different concepts entirely. GHC’s implementation of Generics fall out of the simple observation that all datatypes in Haskell can be written as a combination of a sum of products.
A sum type, is a data structure used to hold a value that could take on several different, but fixed, types. For example:
data Pastry
= Turnover
| Macaroon
| Brownie
| Cookie
A product type, is a data structure used to hold a fixed ordered set of several types. Selecting a single field is called projection.
data Person = Person
{ firstName :: String
, lastName :: String
, age :: Int
, height :: Float
, phoneNumber :: String
, flavor :: String
}
In Haskell all datatypes can be expressed as sums of products:
data Expr
= Add { l :: Expr, r :: Expr }
| Mul { l :: Expr, r :: Expr }
| Sub { l :: Expr, r :: Expr }
| Div { l :: Expr, r :: Expr }
| Number { val :: Int }
During compilation most of the information about the structure of the datatypes is thrown out, by design Haskell erases all type information. Prior to type-checking a phase known as elaboration expands out all record selectors into toplevel functions which extract the named fields of a product.
data Point a b = Point { x :: a, y :: b }
x :: Point a b -> a
x (Point a _) = a
y :: Point a b -> b
y (Point _ b) = b
The rest of the information about products largely gets thrown out after compilation, and the product just get expanded into pattern matching code. For sum types the only information that is kept around is the tag for each constructor of the sum type. For instance Add
is assigned the tag 1, Mul
is assigned 2 etc. In a case statement the only information that is available at runtime is which branch we’re scrutinizing.
So what if consider not tossing out all this information and instead exposed it to our program so that we could write generic logic that can introspect the “structure” of our datatypes.
Since GHC 6.10 we’ve had type families which, among other things, allow us to associate data types with our typeclass. So the structure of our Generic class can have a associated Rep
type which can carry information along with the typeclass.
class Generic a where
type Rep a :: * -> *
from :: a -> (Rep a) x
to :: (Rep a) x -> a
To represent the structure of our datatype we need to set up several datatypes to encode, sums, products, empty branches and various metadata about the names of fields, constructors and their types. All of which have a free parameter p
which is bound to the head of typeclass instance when used in the associated datatype Rep a
.
data V1 p -- Empty
data U1 p = U1 -- ()
data (:+:) f g p = L1 (f p) | R1 (g p) -- Sum
data (:*:) f g p = (f p) :*: (g p) -- Product
newtype K1 i c p = K1 { unK1 :: c } -- a container for a c
newtype M1 i t f p = M1 { unM1 :: f p } -- metadata wrapper
Now we could write this instance by hand for all of our datatypes, and for a simple enumeration it would look like the following:
data Ingredient
= Flour
| Sugar
instance Generic Ingredient where
type Rep Ingredient = M1 D (T_Ingredient ((M1 C (C_Flour U1)) :+: (M1 C (C_Sugar U1))))
from Flour = M1 (L1 (M1 U1))
from Sugar = M1 (R1 (M1 U1))
to (M1 (L1 (M1 U1))) = Flour
to (M1 (R1 (M1 U1))) = Sugar
data T_Ingredient
data C_Flour
data C_Sugar
The instance here is purely mechanical and can be derived from GHC’s internal representation of it’s syntax tree, namely the types GHC.DataCon and GHC.TypeCon. Using the -XDeriveGeneric
extension we can have GHC crank this typeclass out automatically:
{-# LANGUAGE DeriveGeneric #-}
data Ingredient
= Flour
| Sugar
deriving (Generic)
Lest we not handwave away the work that GHC is doing, let’s actually recreate the introspection logic that GHC uses when instantiating a Generic class from a module’s data definitions. Let’s load a module dynamically, intercept the compilation and dump out the internal structure of the it’s datatypes to see how this would be mechanically translated into a typeclass instance.
import GHC
import GHC.Paths as Paths
import Name
import TyCon
import TypeRep
import DataCon
import HscTypes
import Text.Show.Pretty
main :: IO ()
main = do
-- Inside the GHC Monad
rep <- runGhc (Just Paths.libdir) $ do
-- Spin up a GHC compiler environment
dflags <- getSessionDynFlags
setSessionDynFlags dflags
-- Make a dummy module to inject
let mn = mkModuleName "Test"
-- Make a dummy target
addTarget Target {
targetId = TargetModule mn
, targetAllowObjCode = True
, targetContents = Nothing
}
-- Run the GHC pipeline
load LoadAllTargets
modSum <- getModSummary mn
p <- parseModule modSum
t <- typecheckModule p
-- Pluck out the module tycons after we're done type-checking
DesugaredModule tcmod modguts <- desugarModule t
let tycons = mg_tcs modguts
-- Deconstruct all datatypes into their sums-of-products.
return (deconstruct tycons)
putStrLn (ppShow rep)
Now that we have access to GHC’s internal representation of the “module guts” we can write our deconstructor logic. The logic is a slim few hundred lines of mostly ADT munging.
deconstruct :: [TyCon] -> [Data]
deconstruct = fmap go
where
go x
| isProduct x = M1 $ D Datatype
{ dataTypeName = getOccString (tyConName x)
, modName = modString x
, isNewtype = isNewTyCon x
, datatype = Product (mkProduct x)
, recursive = isRecursiveTyCon x
}
| isVoid x = M1 $ D Datatype
{ dataTypeName = getOccString (tyConName x)
, modName = modString x
, isNewtype = isNewTyCon x
, datatype = V1
, recursive = isRecursiveTyCon x
}
| otherwise = M1 $ D Datatype
{ dataTypeName = getOccString (tyConName x)
, modName = modString x
, isNewtype = isNewTyCon x
, datatype = Sum (mkProduct x)
, recursive = isRecursiveTyCon x
}
mkRecord :: TyCon -> [Data]
mkRecord x = concatMap mkRProduct (tyConDataCons x)
mkProduct :: TyCon -> [Data]
mkProduct x = fmap go (tyConDataCons x)
where
go :: DataCon -> Data
go x | isRecord x = Product (mkRProduct x)
go x | isDProduct x = Product (mkDProduct x)
go x = M1 (C (Constructor (conNames x)))
mkDProduct :: DataCon -> [Data]
mkDProduct xs = [K1 (showType x) | x <- dataConOrigArgTys xs]
mkRProduct :: DataCon -> [Data]
mkRProduct x = [M1 (S (Selector (getOccString fld)) ty) | (fld, ty) <- zip (fieldNames x) (mkDProduct x)]
Setting up the dummy module Test.hs
to run our decompilation script:
data PlatonicSolid
= Tetrahedron
| Cube
| Octahedron
| Dodecahedron
| Icosahedron
data Person = Person
{ firstName :: String
, lastName :: String
, age :: Int
, height :: Float
, phoneNumber :: String
, flavor :: String
} deriving (Show)
data T
= T1 { a :: Int, b :: Float }
| T2 { c :: Int, d :: Double }
For PlatonicSolid
we get the representation:
M1
(D Datatype
{ dataTypeName = "PlatonicSolid"
, modName = "Test"
, isNewtype = False
, datatype =
Sum
[ M1 (C Constructor { conName = "Tetrahedron" })
, M1 (C Constructor { conName = "Cube" })
, M1 (C Constructor { conName = "Octahedron" })
, M1 (C Constructor { conName = "Dodecahedron" })
, M1 (C Constructor { conName = "Icosahedron" })
]
, recursive = False
})
For Person
we get the representation:
M1
(D Datatype
{ dataTypeName = "Person"
, modName = "Test"
, isNewtype = False
, datatype =
Product
[ M1 (S Selector { selName = "firstName" } (K1 "String"))
, M1 (S Selector { selName = "lastName" } (K1 "String"))
, M1 (S Selector { selName = "age" } (K1 "Int"))
, M1 (S Selector { selName = "height" } (K1 "Float"))
, M1 (S Selector { selName = "phoneNumber" } (K1 "String"))
, M1 (S Selector { selName = "flavor" } (K1 "String"))
]
, recursive = False
})
For the sum of products T
we get the representation:
M1
(D Datatype
{ dataTypeName = "T"
, modName = "Test"
, isNewtype = False
, datatype =
Sum
[ Product
[ M1 (S Selector { selName = "a" } (K1 "Int"))
, M1 (S Selector { selName = "b" } (K1 "Float"))
]
, Product
[ M1 (S Selector { selName = "c" } (K1 "Int"))
, M1 (S Selector { selName = "d" } (K1 "Double"))
]
]
, recursive = False
})
These data points are then used to generate the Rep instance in the derived Generic instances. So that’s a rough approximation of how -XDeriveGeneric
works under the hood, nothing terribly complicated just book keeping.
From the internal representation we crank out several typeclass instances which store the metadata about the various constructors.
class Datatype d where
datatypeName :: t d f a -> String
moduleName :: t d f a -> String
isNewtype :: t d f a -> Bool
isNewtype _ = False
class Selector s where
selName :: t s f a -> String
class Constructor c where
conName :: t c f a -> String
conFixity :: t c f a -> Fixity
conFixity _ = Prefix
conIsRecord :: t c f a -> Bool
conIsRecord _ = False
For example, for Ingredient
example from before, we’d have several constructor instances automatically generated by which we could query the names from the AST.
type Rep Ingredient = M1 D (T_Ingredient ((M1 C (C_Flour U1)) :+: (M1 C (C_Sugar U1))))
data T_Ingredient
data C_Flour
data C_Sugar
instance Datatype T_Ingredient where
datatypeName _ = "Ingredient"
moduleName _ = "Main"
instance Constructor C_Flour where
conName _ = "Flour"
instance Constructor C_Sugar where
conName _ = "Sugar"
Unlike reflection in languages like Java, Generics are not pushing type information into the runtime. Apart from a dictionary lookup for they are a effectively free abstraction that has no overhead. We’re simply making more information from the compiler manifest in the types during the type-checking phase, all of which gets erased during compilation.
I tried to come up a non-contrived example for illustrating the usefulness of generics, and there are plenty of examples (serializes for JSON, Protocol Buffers, SQL Generation, traversals, command line parsers, etc) that are well-documented elsewhere on the web. So let’s consider an example based on the silly pun in the title of this article, namely cooking typeclasses.
So we have a Pie type, naturally.
data Pie = Pie
{ filling :: Filling
, topping :: Maybe Topping
} deriving (Show, Generic)
data Filling = Apple | Cherry | Pumpkin
deriving (Show, Generic)
data Topping = IceCream | WhipCream
deriving (Show, Generic)
Using generics we’d like to a generate a list of the types of pie that we can put on a menu from the structure of the Haskell types. Records will denote named variations (“filling” vs “topping”) of the menu item, while sum types denote the various options in the variations (“cherry filling” vs “apple filling”).
data Item
= Item Text [Item]
| Variant Text [Item]
| Choice Text
deriving (Show, Generic)
We implement a typeclass with a default signature which gives us the option to manually specify how a type gets converted into a menu item, or fall back on using it’s generic representation to automatically generate it.
class Menu a where
menu :: a -> [Item]
default menu :: (Generic a, GMenu (Rep a)) => a -> [Item]
menu _ = gmenu (Proxy :: Proxy a)
gmenu :: forall a. (Generic a, GMenu (Rep a)) => Proxy a -> [Item]
gmenu _ = gopts (Proxy :: Proxy (Rep a))
Our generic menu operates over various GHC.Generics types to expand out the sums and products into the Item categories that correspond to the menu. The instance for GMenu (K1 R f)
has a Menu constraint which allows manual override for specific datatypes. Since we’re passing around a proxy we’ll have to manually thread the dictionary around sometimes by passing an undefined
cast to the type of the instance we need to resolve.
-- Generic Menu
class GMenu a where
gopts :: Proxy a -> [Item]
-- Datatype
instance GMenu f => GMenu (M1 D x f) where
gopts _ = gopts (Proxy :: Proxy f)
-- Constructor Metadata
instance (GMenu f, Constructor c) => GMenu (M1 C c f) where
gopts x
| conIsRecord (undefined :: t c f a) =
[Item (pack (conName m)) (gopts (Proxy :: Proxy f))]
| otherwise = [Choice (pack (conName m))]
where m = (undefined :: t c f a)
-- Selector Metadata
instance (GMenu f, Selector c) => GMenu (M1 S c f) where
gopts _ = [Variant (pack (selName m)) (gopts (Proxy :: Proxy f))]
where m = (undefined :: t c f a)
-- Constructor Paramater
instance (GMenu (Rep f), Menu f) => GMenu (K1 R f) where
gopts _ = menu (undefined :: f)
-- Sum branch
instance (GMenu a, GMenu b) => GMenu (a :+: b) where
gopts _ = gopts (Proxy :: Proxy a) ++ gopts (Proxy :: Proxy b)
-- Product branch
instance (GMenu a, GMenu b) => GMenu (a :*: b) where
gopts _ = gopts (Proxy :: Proxy a) ++ gopts (Proxy :: Proxy b)
-- Void branch
instance GMenu U1 where
gopts _ = []
Specifically we’ll override the Maybe
type so that it simply expands out to a choice of either “AsIs” of the variant or just the list of choices endowed by the inner parameter.
instance Menu a => Menu (Maybe a) where
menu _ = [Choice (pack "AsIs")] ++ (menu (undefined :: a))
As an example Maybe Topping
expands out into three choices.
menu (a :: Maybe Topping) ~ [Choice "AsIs", Choice "IceCream", Choice "whipCream"]
Using GHC 7.10’s new -XDeriveAnyClass
extension we can actually go back and automatically derive Menu inside the deriving clause.
data Pie = Pie
{ filling :: Filling
, topping :: Maybe Topping
} deriving (Show, Generic, Menu)
Now synthesizing a dictionary for Pie we can get a menu
menu (undefined :: Pie)
And voila:
[ Item
"Pie"
[ Variant
"filling" [ Choice "Apple" , Choice "Cherry" , Choice "Pumpkin" ]
, Variant
"topping"
[ Choice "AsIs" , Choice "IceCream" , Choice "WhipCream" ]
]
]
Since our logic is datatype generic, any Haskell we can write down can be automatically translated to a Menu just by deriving Menu
. So now we can a new Crisp
desert (my favorite!) and we get everything for free!
data Crisp = Crisp
{ contents :: Filling
, temperature :: Temperature
} deriving (Show, Generic, Menu)
data Temperature = Warm | Cold
deriving (Show, Generic, Menu)
-- Add an instance for a pair of menu items. That expands into multiple items.
instance (Menu a, Menu b) => Menu (a,b) where
menu _ = menu (undefined :: a) ++ menu (undefined :: b)
And we can generate the composite menu of both deserts:
menu (undefined :: (Pie, Crisp))
[ Item
"Pie"
[ Variant
"filling" [ Choice "Apple" , Choice "Cherry" , Choice "Pumpkin" ]
, Variant
"topping"
[ Choice "AsIs" , Choice "IceCream" , Choice "WhipCream" ]
]
, Item
"Crisp"
[ Variant
"contents" [ Choice "Apple" , Choice "Cherry" , Choice "Pumpkin" ]
, Variant "temperature" [ Choice "Warm" , Choice "Cold" ]
]
]
So that’s generics. One of the best goto examples of how an expressive type system and a few clever compiler hooks can make programmers lives easier by cooking our boilerplate for us and giving tastier more correct code.


Shop now at the Apple Store online with many great ways to buy. Get free, no-contact delivery, Specialist help and more >
iPhone 12
Blast past fast.
From Rs 69900* before trade-in
Learn More > Buy >
Buy a Mac or iPad for
university. Get AirPods.1
Save with education pricing >
iPad Pro
Supercharged by the Apple M1 chip.
Learn More > Buy >
iMac
Say Hello
Learn More > Buy >
Apple at work
11 Reasons
Mac means business
Learn More >
Introducing Spatial Audio with Dolby atmos.
Listen Now 2


Hi guys, First, Excuse my bad english =\ My main language is Spanish
And 1 of the rulz is only English :s
Anyway, Here my question maybe idea
can be possible put the chat window a little more bigger ?
Or DICE have to do it ? sometimes i'm in a party and feel like the window is to small
and i have to scroll up to read -.-!
BTW, I<3BBLOG
José Ortega
Global Moderator
I dont know if this is possible, but i think it's more for a plugin thingy than a BBLog feature. you should wait for Roland's response.
No se si sea posible, pero seria más apropiado que se cree un plugin en vez de integrarlo en BBLog. Tendrás que esperar una respuesta de Roland.
Topicstarter
No_Other_xD
ok =\ thanks
BrainFooLong
Administrator
Sorry, BBLog doesn't do layout fixes/improvements in general.
Maybe a theme or plugin developer will help you.


In this section you will find a tutorial video for HangMan, as well as its rules.
In Single-Player mode, the objective of the game is to guess a certain word, letter by letter ,in a space of time. For every wrong letter you lose one "life", and after a limited number of lives lost, the game ends. The greater the difficulty chosen, the lower the playing time.
In Multi-Player mode, two players choose one word each for the opponent to guess. After the choice, both players play side by side at the same time, with the same rules of the Single-Player mode.


Hey, welcome to our website. We built this to showcase our final project for Google's Applied Machine Learning Intensive (AMLI). Our project idea was to develop a website that would display real tweets next to fake ones and ask the user to guess the real one. We wanted to use machine learning to generate the text, and our first idea was to use a LSTM based neural network. Our results were mixed. We tried word level and character level modeling, but both produced subpar results. We were struggling with the development of our own models, so we looked to see if we were reinventing the wheel, we where!
In February 2019 the ML community saw a great advance in language modeling with Open ai's release of their gpt-2 117M model.Although this isn't the full model they used in their paper, the results are still impressive. Thanks to nsheppered's addtions to the original git repository, we are were able to train this state of the art model on our twitter data, producing some convincing fake tweets.
Producing fake tweets is nothing new, in our research we came across many examples, but the only one that really stuck out to us was DeepDrumpf created by @BradHayes. We were unable to speak with Mr. Hayes, so we don't know what he uses, but his results are impressive, his work was our original performance benchmark. All the examples we came across were utilizing a neural net with LSTM or GRU nodes. We believe this to be the first application fo GPT-2 117M to the tweet generation problem. Of course this is a cute application of some powerful technology, but we believe it to be an eye opening experiment with the possibilities of misinformation.


Functions and defines to create and manipulate instruction operands. More...
Functions and defines to create and manipulate instruction operands.
| #define DR_REG_LAST_ENUM DR_REG_YMM15 |
Last value of register enums
| #define DR_REG_LAST_VALID_ENUM DR_REG_YMM15 |
Last valid register enum value. Note: DR_REG_INVALID is now smaller than this value.
| #define DR_REG_START_16 DR_REG_AX |
Start of 16-bit general register enum values
| #define DR_REG_START_32 DR_REG_EAX |
Start of 32-bit general register enum values
| #define DR_REG_START_64 DR_REG_RAX |
Start of 64-bit general register enum values
| #define DR_REG_START_8 DR_REG_AL |
Start of 8-bit general register enum values
| #define DR_REG_START_8HL DR_REG_AL |
Start of 8-bit high-low register enum values
| #define DR_REG_START_CR DR_REG_CR0 |
Start of control register enum values
| #define DR_REG_START_DR DR_REG_DR0 |
Start of debug register enum values
| #define DR_REG_START_FLOAT DR_REG_ST0 |
Start of floating-point-register enum values
| #define DR_REG_START_MMX DR_REG_MM0 |
Start of mmx register enum values
| #define DR_REG_START_SEGMENT DR_SEG_ES |
Start of segment register enum values
| #define DR_REG_START_x64_8 DR_REG_SPL |
Start of 8-bit x64-only register enum values
| #define DR_REG_START_x86_8 DR_REG_AH |
Start of 8-bit x86-only register enum values
| #define DR_REG_START_XMM DR_REG_XMM0 |
Start of xmm register enum values
| #define DR_REG_START_YMM DR_REG_YMM0 |
Start of ymm register enum values
| #define DR_REG_STOP_16 DR_REG_R15W |
End of 16-bit general register enum values
| #define DR_REG_STOP_32 DR_REG_R15D |
End of 32-bit general register enum values
| #define DR_REG_STOP_64 DR_REG_R15 |
End of 64-bit general register enum values
| #define DR_REG_STOP_8 DR_REG_DIL |
End of 8-bit general register enum values
| #define DR_REG_STOP_8HL DR_REG_BH |
End of 8-bit high-low register enum values
| #define DR_REG_STOP_CR DR_REG_CR15 |
End of control register enum values
| #define DR_REG_STOP_DR DR_REG_DR15 |
End of debug register enum values
| #define DR_REG_STOP_FLOAT DR_REG_ST7 |
End of floating-point-register enum values
| #define DR_REG_STOP_MMX DR_REG_MM7 |
End of mmx register enum values
| #define DR_REG_STOP_SEGMENT DR_SEG_GS |
End of segment register enum values
| #define DR_REG_STOP_x64_8 DR_REG_DIL |
Stop of 8-bit x64-only register enum values
| #define DR_REG_STOP_x86_8 DR_REG_BH |
Stop of 8-bit x86-only register enum values
| #define DR_REG_STOP_XMM DR_REG_XMM15 |
End of xmm register enum values
| #define DR_REG_STOP_YMM DR_REG_YMM15 |
End of ymm register enum values
| #define DR_REG_XAX DR_REG_RAX |
Platform-independent way to refer to rax/eax.
| #define DR_REG_XBP DR_REG_RBP |
Platform-independent way to refer to rbp/ebp.
| #define DR_REG_XBX DR_REG_RBX |
Platform-independent way to refer to rbx/ebx.
| #define DR_REG_XCX DR_REG_RCX |
Platform-independent way to refer to rcx/ecx.
| #define DR_REG_XDI DR_REG_RDI |
Platform-independent way to refer to rdi/edi.
| #define DR_REG_XDX DR_REG_RDX |
Platform-independent way to refer to rdx/edx.
| #define DR_REG_XSI DR_REG_RSI |
Platform-independent way to refer to rsi/esi.
| #define DR_REG_XSP DR_REG_RSP |
Platform-independent way to refer to rsp/esp.
| #define OPSZ_bound OPSZ_8_short4 |
Operand size for bound memory reference.
| #define OPSZ_call OPSZ_ret |
Operand size for push portion of call.
| #define OPSZ_clflush OPSZ_1 |
Operand size for clflush memory reference.
| #define OPSZ_fldenv OPSZ_28_short14 |
Operand size for fldenv memory reference.
| #define OPSZ_fnsave OPSZ_108_short94 |
Operand size for fnsave memory reference.
| #define OPSZ_fnstenv OPSZ_28_short14 |
Operand size for fnstenv memory reference.
| #define OPSZ_frstor OPSZ_108_short94 |
Operand size for frstor memory reference.
| #define OPSZ_fxrstor OPSZ_512 |
Operand size for fxrstor memory reference.
| #define OPSZ_fxsave OPSZ_512 |
Operand size for fxsave memory reference.
| #define OPSZ_invlpg OPSZ_0 |
Operand size for invlpg memory reference.
| #define OPSZ_lea OPSZ_0 |
Operand size for lea memory reference.
| #define OPSZ_lgdt OPSZ_6x10 |
Operand size for lgdt memory reference.
| #define OPSZ_lidt OPSZ_6x10 |
Operand size for lidt memory reference.
| #define OPSZ_maskmovdqu OPSZ_16 |
Operand size for maskmovdqu memory reference.
| #define OPSZ_maskmovq OPSZ_8 |
Operand size for maskmovq memory reference.
| #define OPSZ_prefetch OPSZ_1 |
Operand size for prefetch memory references.
| #define OPSZ_PTR OPSZ_8 |
Operand size for pointer values.
| #define OPSZ_ret OPSZ_4x8_short2xi8 |
Operand size for ret instruction.
| #define OPSZ_sgdt OPSZ_6x10 |
Operand size for sgdt memory reference.
| #define OPSZ_sidt OPSZ_6x10 |
Operand size for sidt memory reference.
| #define OPSZ_STACK OPSZ_8 |
Operand size for stack push/pop operand sizes.
| #define OPSZ_VARSTACK OPSZ_4x8_short2 |
Operand size for prefix-varying stack push/pop operand sizes.
| #define OPSZ_xlat OPSZ_1 |
Operand size for xlat memory reference.
| anonymous enum |
| anonymous enum |
| const char* get_register_name | ( | reg_id_t | reg | ) |
Assumes that reg
is a DR_REG_ 32-bit register constant. Returns the string name for reg
.
| app_pc opnd_compute_address | ( | opnd_t | opnd, | |
| dr_mcontext_t * | mc | |||
| ) |
Returns the effective address of opnd
, computed using the passed-in register values. If opnd
is a far address, ignores that aspect except for TLS references on Windows (fs: for 32-bit, gs: for 64-bit) or typical fs: or gs: references on Linux. For far addresses the calling thread's segment selector is used. mc->flags
must include DR_MC_CONTROL and DR_MC_INTEGER.
| opnd_t opnd_create_abs_addr | ( | void * | addr, | |
| opnd_size_t | data_size | |||
| ) |
Returns a memory reference operand that refers to the address addr
. The operand has data size data_size
(must be a OPSZ_ constant).
If addr
<= 2^32 (which is always true in 32-bit mode), this routine is equivalent to opnd_create_base_disp(DR_REG_NULL, DR_REG_NULL, 0, (int)addr, data_size).
Otherwise, this routine creates a separate operand type with an absolute 64-bit memory address. Note that such an operand can only be used as a load or store from or to the rax register.
| opnd_t opnd_create_base_disp | ( | reg_id_t | base_reg, | |
| reg_id_t | index_reg, | |||
| int | scale, | |||
| int | disp, | |||
| opnd_size_t | data_size | |||
| ) |
Returns a memory reference operand that refers to the address:
or, in other words,
The operand has data size data_size (must be a OPSZ_ constant). Both base_reg
and index_reg
must be DR_REG_ constants. scale
must be either 1, 2, 4, or 8.
| opnd_t opnd_create_base_disp_ex | ( | reg_id_t | base_reg, | |
| reg_id_t | index_reg, | |||
| int | scale, | |||
| int | disp, | |||
| opnd_size_t | size, | |||
| bool | encode_zero_disp, | |||
| bool | force_full_disp, | |||
| bool | disp_short_addr | |||
| ) |
Returns a memory reference operand that refers to the address:
or, in other words,
The operand has data size data_size
(must be a OPSZ_ constant). Both base_reg
and index_reg
must be DR_REG_ constants. scale
must be either 1, 2, 4, or 8. Gives control over encoding optimizations:
encode_zero_disp
, a zero value for disp will not be omitted;force_full_disp
, a small value for disp will not occupy only one byte.disp_short_addr
, short (16-bit for 32-bit mode, 32-bit for 64-bit mode) addressing will be used (note that this normally only needs to be specified for an absolute address; otherwise, simply use the desired short registers for base and/or index).(Both of those are false when using opnd_create_base_disp()).
| opnd_t opnd_create_far_abs_addr | ( | reg_id_t | seg, | |
| void * | addr, | |||
| opnd_size_t | data_size | |||
| ) |
Returns a memory reference operand that refers to the address seg:
addr
. The operand has data size data_size
(must be a OPSZ_ constant).
If addr
<= 2^32 (which is always true in 32-bit mode), this routine is equivalent to opnd_create_far_base_disp(seg, DR_REG_NULL, DR_REG_NULL, 0, (int)addr, data_size).
Otherwise, this routine creates a separate operand type with an absolute 64-bit memory address. Note that such an operand can only be used as a load or store from or to the rax register.
| opnd_t opnd_create_far_base_disp | ( | reg_id_t | seg, | |
| reg_id_t | base_reg, | |||
| reg_id_t | index_reg, | |||
| int | scale, | |||
| int | disp, | |||
| opnd_size_t | data_size | |||
| ) |
Returns a far memory reference operand that refers to the address:
or, in other words,
The operand has data size data_size
(must be a OPSZ_ constant). seg
must be a DR_SEG_ constant. Both base_reg
and index_reg
must be DR_REG_ constants. scale
must be either 1, 2, 4, or 8.
| opnd_t opnd_create_far_base_disp_ex | ( | reg_id_t | seg, | |
| reg_id_t | base_reg, | |||
| reg_id_t | index_reg, | |||
| int | scale, | |||
| int | disp, | |||
| opnd_size_t | size, | |||
| bool | encode_zero_disp, | |||
| bool | force_full_disp, | |||
| bool | disp_short_addr | |||
| ) |
Returns a far memory reference operand that refers to the address:
or, in other words,
The operand has data size data_size
(must be a OPSZ_ constant). seg
must be a DR_SEG_ constant. Both base_reg
and index_reg
must be DR_REG_ constants. scale must be either 1, 2, 4, or 8. Gives control over encoding optimizations:
encode_zero_disp
, a zero value for disp will not be omitted;force_full_disp
, a small value for disp will not occupy only one byte.disp_short_addr
, short (16-bit for 32-bit mode, 32-bit for 64-bit mode) addressing will be used (note that this normally only needs to be specified for an absolute address; otherwise, simply use the desired short registers for base and/or index).(Both of those are false when using opnd_create_far_base_disp()).
Returns a far instr_t pointer address with value seg_selector:instr
. seg_selector
is a segment selector, not a DR_SEG_ constant.
| opnd_t opnd_create_far_pc | ( | ushort | seg_selector, | |
| app_pc | pc | |||
| ) |
Returns a far program address operand with value seg_selector:pc
. seg_selector
is a segment selector, not a DR_SEG_ constant.
| opnd_t opnd_create_far_rel_addr | ( | reg_id_t | seg, | |
| void * | addr, | |||
| opnd_size_t | data_size | |||
| ) |
Returns a memory reference operand that refers to the address seg
: addr
, but will be encoded as a pc-relative address. It is up to the caller to ensure that the resulting address is reachable via a 32-bit signed displacement from the next instruction at emit time.
DR guarantees that all of its code caches and heap are within the same 2GB memory region. DR also loads client libraries within 32-bit reachability of its code caches and heap. This means that any static data or code in a client library, or any data allocated using DR's API, is guaranteed to be reachable from code cache code.
The operand has data size data_size
(must be a OPSZ_ constant).
To represent a 32-bit address (i.e., what an address size prefix indicates), simply zero out the top 32 bits of the address before passing it to this routine.
| opnd_t opnd_create_immed_float | ( | float | f | ) |
Returns an immediate float operand with value f
. The caller's code should use proc_save_fpstate() or be inside a clean call that has requested to preserve the floating-point state.
| opnd_t opnd_create_immed_int | ( | ptr_int_t | i, | |
| opnd_size_t | data_size | |||
| ) |
Returns an immediate integer operand with value i
and size data_size
; data_size
must be a OPSZ_ constant.
| opnd_t opnd_create_null | ( | void | ) |
Returns an empty operand.
| opnd_t opnd_create_pc | ( | app_pc | pc | ) |
Returns a program address operand with value pc
.
| opnd_t opnd_create_reg | ( | reg_id_t | r | ) |
Returns a register operand (r
must be a DR_REG_ constant).
| opnd_t opnd_create_rel_addr | ( | void * | addr, | |
| opnd_size_t | data_size | |||
| ) |
Returns a memory reference operand that refers to the address addr
, but will be encoded as a pc-relative address. At emit time, if addr
is out of reach of a 32-bit signed displacement from the next instruction, encoding will fail.
DR guarantees that all of its code caches and heap are within the same 2GB memory region. DR also loads client libraries within 32-bit reachability of its code caches and heap. This means that any static data or code in a client library, or any data allocated using DR's API, is guaranteed to be reachable from code cache code.
The operand has data size data_size (must be a OPSZ_ constant).
To represent a 32-bit address (i.e., what an address size prefix indicates), simply zero out the top 32 bits of the address before passing it to this routine.
Returns true iff def
, considered as a write, affects use
. Is conservative, so if both def
and use
are memory references, will return true unless it can disambiguate them based on their registers and displacement.
| void opnd_disassemble | ( | void * | drcontext, | |
| opnd_t | opnd, | |||
| file_t | outfile | |||
| ) |
Prints the operand opnd
to file outfile
. The default is to use AT&T-style syntax, unless the -syntax_intel runtime option is specified.
| size_t opnd_disassemble_to_buffer | ( | void * | drcontext, | |
| opnd_t | opnd, | |||
| char * | buf, | |||
| size_t | bufsz | |||
| ) |
Prints the operand opnd
to the buffer buf
. Always null-terminates, and will not print more than bufsz
characters, which includes the final null character. Returns the number of characters printed, not including the final null. The default is to use AT&T-style syntax, unless the -syntax_intel runtime option is specified.
| void* opnd_get_addr | ( | opnd_t | opnd | ) |
Assumes opnd
is a (near or far) absolute or pc-relative memory reference, or a base+disp memory reference with no base or index register. Returns opnd's
absolute address (which will be pc-relativized on encoding for pc-relative memory references).
| reg_id_t opnd_get_base | ( | opnd_t | opnd | ) |
Assumes opnd
is a (near or far) base+disp memory reference. Returns the base register (a DR_REG_ constant).
| int opnd_get_disp | ( | opnd_t | opnd | ) |
Assumes opnd
is a (near or far) base+disp memory reference. Returns the displacement.
| float opnd_get_immed_float | ( | opnd_t | opnd | ) |
Assumes opnd
is an immediate float and returns its value. The caller's code should use proc_save_fpstate() or be inside a clean call that has requested to preserve the floating-point state.
| reg_id_t opnd_get_index | ( | opnd_t | opnd | ) |
Assumes opnd
is a (near or far) base+disp memory reference. Returns the index register (a DR_REG_ constant).
| app_pc opnd_get_pc | ( | opnd_t | opnd | ) |
Assumes opnd
is a (near or far) program address, returns its value.
| reg_id_t opnd_get_reg | ( | opnd_t | opnd | ) |
Assumes opnd
is a register operand. Returns the register it refers to (a DR_REG_ constant).
| reg_id_t opnd_get_reg_used | ( | opnd_t | opnd, | |
| int | index | |||
| ) |
Used in conjunction with opnd_num_regs_used(), this routine can be used to iterate through all registers used by opnd
. The index values begin with 0 and proceed through opnd_num_regs_used(opnd)-1.
| int opnd_get_scale | ( | opnd_t | opnd | ) |
Assumes opnd
is a (near or far) base+disp memory reference. Returns the scale.
| reg_id_t opnd_get_segment | ( | opnd_t | opnd | ) |
Assumes opnd
is a (near or far) memory reference of any type. Returns opnd's
segment (a DR_SEG_ constant), or DR_REG_NULL if it is a near memory reference.
| ushort opnd_get_segment_selector | ( | opnd_t | opnd | ) |
Assumes opnd
is a far program address. Returns opnd's
segment, a segment selector (not a DR_SEG_ constant).
| opnd_size_t opnd_get_size | ( | opnd_t | opnd | ) |
Return the data size of opnd
as a OPSZ_ constant. Assumes opnd
is a register, immediate integer, or memory reference. If opnd
is a register returns the result of opnd_reg_get_size() called on the DR_REG_ constant. Returns OPSZ_NA if opnd
does not have a valid size.
| bool opnd_is_abs_addr | ( | opnd_t | opnd | ) |
Returns true iff opnd
is a (near or far) absolute address operand. Returns true for both base-disp operands with no base or index and 64-bit non-base-disp absolute address operands.
| bool opnd_is_base_disp | ( | opnd_t | opnd | ) |
Returns true iff opnd
is a (near or far) base+disp memory reference operand.
| bool opnd_is_disp_encode_zero | ( | opnd_t | opnd | ) |
Assumes opnd
is a (near or far) base+disp memory reference; returns whether encode_zero_disp has been specified for opnd
.
| bool opnd_is_disp_force_full | ( | opnd_t | opnd | ) |
Assumes opnd
is a (near or far) base+disp memory reference; returns whether force_full_disp has been specified for opnd
.
| bool opnd_is_disp_short_addr | ( | opnd_t | opnd | ) |
Assumes opnd
is a (near or far) base+disp memory reference; returns whether disp_short_addr has been specified for opnd
.
| bool opnd_is_far_abs_addr | ( | opnd_t | opnd | ) |
Returns true iff opnd
is a far absolute address operand. Returns true for both base-disp operands with no base or index and 64-bit non-base-disp absolute address operands.
| bool opnd_is_far_base_disp | ( | opnd_t | opnd | ) |
Returns true iff opnd
is a far base+disp memory reference operand.
| bool opnd_is_far_instr | ( | opnd_t | opnd | ) |
Returns true iff opnd
is a far instr_t pointer address operand.
| bool opnd_is_far_memory_reference | ( | opnd_t | opnd | ) |
Returns true iff opnd
is a far memory reference operand of any type: base-disp, absolute address, or pc-relative address.
| bool opnd_is_far_pc | ( | opnd_t | opnd | ) |
Returns true iff opnd
is a far program address operand.
| bool opnd_is_far_rel_addr | ( | opnd_t | opnd | ) |
Returns true iff opnd
is a far pc-relative memory reference operand.
| bool opnd_is_immed | ( | opnd_t | opnd | ) |
Returns true iff opnd
is an immediate (integer or float) operand.
| bool opnd_is_immed_float | ( | opnd_t | opnd | ) |
Returns true iff opnd
is an immediate float operand.
| bool opnd_is_immed_int | ( | opnd_t | opnd | ) |
Returns true iff opnd
is an immediate integer operand.
| bool opnd_is_instr | ( | opnd_t | opnd | ) |
Returns true iff opnd
is a (near or far) instr_t pointer address operand.
| bool opnd_is_memory_reference | ( | opnd_t | opnd | ) |
Returns true iff opnd
is a (near or far) memory reference operand of any type: base-disp, absolute address, or pc-relative address.
| bool opnd_is_near_abs_addr | ( | opnd_t | opnd | ) |
Returns true iff opnd
is a near (i.e., default segment) absolute address operand. Returns true for both base-disp operands with no base or index and 64-bit non-base-disp absolute address operands.
| bool opnd_is_near_base_disp | ( | opnd_t | opnd | ) |
Returns true iff opnd
is a near (i.e., default segment) base+disp memory reference operand.
| bool opnd_is_near_instr | ( | opnd_t | opnd | ) |
Returns true iff opnd
is a near instr_t pointer address operand.
| bool opnd_is_near_memory_reference | ( | opnd_t | opnd | ) |
Returns true iff opnd
is a near memory reference operand of any type: base-disp, absolute address, or pc-relative address.
| bool opnd_is_near_pc | ( | opnd_t | opnd | ) |
Returns true iff opnd
is a near (i.e., default segment) program address operand.
| bool opnd_is_near_rel_addr | ( | opnd_t | opnd | ) |
Returns true iff opnd
is a near (i.e., default segment) pc-relative memory reference operand.
| bool opnd_is_null | ( | opnd_t | opnd | ) |
Returns true iff opnd
is an empty operand.
| bool opnd_is_pc | ( | opnd_t | opnd | ) |
Returns true iff opnd
is a (near or far) program address operand.
| bool opnd_is_reg | ( | opnd_t | opnd | ) |
Returns true iff opnd
is a register operand.
| bool opnd_is_reg_32bit | ( | opnd_t | opnd | ) |
Returns true iff opnd
is a register operand that refers to a 32-bit general-purpose register.
| bool opnd_is_reg_64bit | ( | opnd_t | opnd | ) |
Returns true iff opnd
is a register operand that refers to a 64-bit general-purpose register.
| bool opnd_is_reg_pointer_sized | ( | opnd_t | opnd | ) |
Returns true iff opnd
is a register operand that refers to a pointer-sized general-purpose register.
| bool opnd_is_rel_addr | ( | opnd_t | opnd | ) |
Returns true iff opnd
is a (near or far) pc-relative memory reference operand.
| int opnd_num_regs_used | ( | opnd_t | opnd | ) |
Returns the number of registers referred to by opnd
. This will only be non-zero for register operands and memory references.
| bool opnd_replace_reg | ( | opnd_t * | opnd, | |
| reg_id_t | old_reg, | |||
| reg_id_t | new_reg | |||
| ) |
Assumes that both old_reg
and new_reg
are DR_REG_ constants. Replaces all occurrences of old_reg
in *opnd
with new_reg
.
Returns true iff op1
and op2
are indistinguishable. If either uses variable operand sizes, the default size is assumed.
Returns true iff op1
and op2
are both memory references and they are indistinguishable, ignoring data size.
| void opnd_set_disp | ( | opnd_t * | opnd, | |
| int | disp | |||
| ) |
Set the displacement of a memory reference operand opnd
to disp
.
| void opnd_set_disp_ex | ( | opnd_t * | opnd, | |
| int | disp, | |||
| bool | encode_zero_disp, | |||
| bool | force_full_disp, | |||
| bool | disp_short_addr | |||
| ) |
Set the displacement and encoding controls of a memory reference operand:
encode_zero_disp
, a zero value for disp
will not be omitted;force_full_disp
, a small value for disp
will not occupy only one byte.disp_short_addr
, short (16-bit for 32-bit mode, 32-bit for 64-bit mode) addressing will be used (note that this normally only needs to be specified for an absolute address; otherwise, simply use the desired short registers for base and/or index). | void opnd_set_size | ( | opnd_t * | opnd, | |
| opnd_size_t | newsize | |||
| ) |
Sets the data size of opnd
. Assumes opnd
is an immediate integer or a memory reference.
Returns true iff there exists some register that is referred to (directly or overlapping) by both op1
and op2
.
Shrinks all 32-bit registers in opnd
to their 16-bit versions. Also shrinks the size of immediate integers and memory references from OPSZ_4 to OPSZ_2.
Shrinks all 64-bit registers in opnd
to their 32-bit versions. Also shrinks the size of immediate integers and memory references from OPSZ_8 to OPSZ_4.
| uint opnd_size_in_bytes | ( | opnd_size_t | size | ) |
Assumes size
is a OPSZ_ or a DR_REG_ constant. If size
is a DR_REG_ constant, first calls reg_get_size(size
) to get a OPSZ_ constant. Returns the number of bytes the OPSZ_ constant represents. If OPSZ_ is a variable-sized size, returns the default size, which may or may not match the actual size decided up on at encoding time (that final size depends on other operands).
| bool opnd_uses_reg | ( | opnd_t | opnd, | |
| reg_id_t | reg | |||
| ) |
Assumes that reg
is a DR_REG_ constant. Returns true iff opnd
refers to reg directly or refers to a register that overlaps reg
(e.g., DR_REG_AX overlaps DR_REG_EAX).
| reg_id_t reg_32_to_16 | ( | reg_id_t | reg | ) |
Assumes that reg
is a DR_REG_ 32-bit register constant. Returns the 16-bit version of reg
.
| reg_id_t reg_32_to_64 | ( | reg_id_t | reg | ) |
Assumes that reg
is a DR_REG_ 32-bit register constant. Returns the 64-bit version of reg
.
| reg_id_t reg_32_to_8 | ( | reg_id_t | reg | ) |
Assumes that reg
is a DR_REG_ 32-bit register constant. Returns the 8-bit version of reg
(the least significant byte: DR_REG_AL instead of DR_REG_AH if passed DR_REG_EAX, e.g.). For 32-bit DR builds, returns DR_REG_NULL if passed DR_REG_ESP, DR_REG_EBP, DR_REG_ESI, or DR_REG_EDI.
| reg_id_t reg_32_to_opsz | ( | reg_id_t | reg, | |
| opnd_size_t | sz | |||
| ) |
Assumes that reg
is a DR_REG_ 32-bit register constant. If sz
== OPSZ_2, returns the 16-bit version of reg
. For 64-bit versions of this library, if sz
== OPSZ_8, returns the 64-bit version of reg
.
| reg_id_t reg_64_to_32 | ( | reg_id_t | reg | ) |
Assumes that reg
is a DR_REG_ 64-bit register constant. Returns the 32-bit version of reg
.
| byte reg_get_bits | ( | reg_id_t | reg | ) |
Assumes that reg
is a DR_REG_ constant. Returns reg's
representation as 3 bits in a modrm byte (the 3 bits are the lower-order bits in the return value).
| opnd_size_t reg_get_size | ( | reg_id_t | reg | ) |
Assumes that reg
is a DR_REG_ constant. Returns the OPSZ_ constant corresponding to the register size. Returns OPSZ_NA if reg is not a DR_REG_ constant.
| reg_t reg_get_value | ( | reg_id_t | reg, | |
| dr_mcontext_t * | mc | |||
| ) |
Returns the value of the register reg
, selected from the passed-in register values. Supports only general-purpose registers. mc->flags
must include DR_MC_CONTROL and DR_MC_INTEGER.
| bool reg_is_32bit | ( | reg_id_t | reg | ) |
Assumes that reg
is a DR_REG_ constant. Returns true iff it refers to a 32-bit general-purpose register.
| bool reg_is_64bit | ( | reg_id_t | reg | ) |
Assumes that reg
is a DR_REG_ constant. Returns true iff it refers to a 64-bit general-purpose register.
| bool reg_is_extended | ( | reg_id_t | reg | ) |
Returns true iff reg
refers to an extended register only available in 64-bit mode and not in 32-bit mode (e.g., R8-R15, XMM8-XMM15, etc.)
| bool reg_is_fp | ( | reg_id_t | reg | ) |
Assumes that reg
is a DR_REG_ constant. Returns true iff it refers to a floating-point register.
| bool reg_is_gpr | ( | reg_id_t | reg | ) |
Assumes that reg
is a DR_REG_ constant. Returns true iff it refers to a General Purpose Register, i.e., rax, rcx, rdx, rbx, rsp, rbp, rsi, rdi, or a subset.
| bool reg_is_mmx | ( | reg_id_t | reg | ) |
Assumes that reg
is a DR_REG_ constant. Returns true iff it refers to an mmx (64-bit) register.
| bool reg_is_pointer_sized | ( | reg_id_t | reg | ) |
Assumes that reg
is a DR_REG_ constant. Returns true iff it refers to a pointer-sized general-purpose register.
| bool reg_is_segment | ( | reg_id_t | reg | ) |
Assumes that reg
is a DR_REG_ constant. Returns true iff it refers to a segment (i.e., it's really a DR_SEG_ constant).
| bool reg_is_xmm | ( | reg_id_t | reg | ) |
Assumes that reg
is a DR_REG_ constant. Returns true iff it refers to an xmm (128-bit SSE/SSE2) register or a ymm (256-bit multimedia) register.
| bool reg_is_ymm | ( | reg_id_t | reg | ) |
Assumes that reg
is a DR_REG_ constant. Returns true iff it refers to a ymm (256-bit multimedia) register.
| bool reg_overlap | ( | reg_id_t | r1, | |
| reg_id_t | r2 | |||
| ) |
Assumes that r1
and r2
are both DR_REG_ constants. Returns true iff r1's
register overlaps r2's
register (e.g., if r1
== DR_REG_AX and r2
== DR_REG_EAX).
| int reg_parameter_num | ( | reg_id_t | reg | ) |
Assumes that reg
is a DR_REG_ register constant. If reg is used as part of the calling convention, returns which parameter ordinal it matches (0-based); otherwise, returns -1.
| void reg_set_value | ( | reg_id_t | reg, | |
| dr_mcontext_t * | mc, | |||
| reg_t | value | |||
| ) |
Sets the register reg
in the passed in mcontext mc
to value
. mc->flags
must include DR_MC_CONTROL and DR_MC_INTEGER.
| reg_id_t reg_to_pointer_sized | ( | reg_id_t | reg | ) |
Assumes that reg
is a DR_REG_ 32-bit register constant. Returns the pointer-sized version of reg
.


"{{item.quote}} "
-{{item.quoteName}}-{{item.jobTitle}}{{item.dateAdded|date}}
This quote was submitted by: {{item.quoteSubmitter}} {{item.dateAdded|quoteLife}}
Cumulative votes: {{item.upvoteValue + item.downvoteValue}}


Simpson carries South Dakota over Hawaii 81-75
HONOLULU (AP) - Triston Simpson scored 22 points and South Dakota beat Hawaii 81-75 on Sunday night at the Outrigger Resorts Rainbow Classic.
Stanley Umude added 21 points and seven rebounds for the Coyotes (2-0) and Hunter Goodrick had 15 points and seven rebounds.
Eddie Stansberry had 21 points for the Rainbow Warriors (1-1). Samuta Avea added 14 points, hitting 4 of 6 from 3-point range, and Drew Buggs had 12 points and five assists.
Stansberry hit a 3-pointer sparks a 17-8 spurt that trimmed Hawaii’s deficit to 72-70 when Zigmars Raimo made a layup with 1:08 to play but the Coyotes made 9 of 10 from the free-throw line from there to seal it.
South Dakota matches up against Florida A&M; on Monday. Hawaii plays Pacific at home on Monday.
For more AP college basketball coverage: https://apnews.com/Collegebasketball and http://twitter.com/AP_Top25
Elements of this story were generated by Automated Insights, http://www.automatedinsights.com/ap, using data from STATS LLC, https://www.stats.com
Sign up for Daily Newsletters
Copyright © 2019 The Washington Times, LLC.
Please read our comment policy before commenting.


LICENSE AGREEMENT FOR ULTRASCHALL SOFTWARE:
Copyright (c) 2015 Ralf Stockmann, Malte Dreschert, Daniel Lindenfelser, Andreas Pieper, Tim Pritlove, Heiko Panjas
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
 IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
LICENSE AGREEMENT FOR SWS EXTENSION:
Copyright (c) 2015 Tim Payne (SWS), Jeffos (S&M), Xenakios, Fingers, P Bourdon, Adam Wathan, Shane StClair, Philip S. Considine, Dominik Martin Drzic.
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
 IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.


It's a system for users to manager their own attribute and sign others' attribute!
Attribute
Any thing can be an attribute and store into the space which is already, but here are some rules:
1. Attribute can't change name and be deleted, think before create.
2. Every attribute must be updated in 30 days.
Attribute name:
Data:
You can see other's attribute if you know his/her Ethereum address.
Key in the address and ID number, you can get information.
ps. ID number is the sequence when the attribute is created.
Target Address:
Attribute ID:
The "Extend Time" button is used to extend time of Attribute.
If the owner didn't extend, the attribute can't be access, and signers' deposit will be confiscated.
If you want to modify your attribute, you can press "Update Attribute" button, but should beware, the attribute will be reset.(Signatures will be cleared)
Attribute ID:
Update Data:
Sign
You can "sign" the other's attribute to represent your trust, but you must send deposit to sign it.
But if the attribute didn't update in time, you won't get your money back!
You must be sure the address you sign is worth to trust.
You can cancel your signatures, and if the attribute still work well at that time,
you can get your deposit back.


Introducing Gelly: Graph Processing with Apache Flink
August 24, 2015 -This blog post introduces Gelly, Apache Flink’s graph-processing API and library. Flink’s native support for iterations makes it a suitable platform for large-scale graph analytics. By leveraging delta iterations, Gelly is able to map various graph processing models such as vertex-centric or gather-sum-apply to Flink dataflows.
Gelly allows Flink users to perform end-to-end data analysis in a single system. Gelly can be seamlessly used with Flink’s DataSet API, which means that pre-processing, graph creation, analysis, and post-processing can be done in the same application. At the end of this post, we will go through a step-by-step example in order to demonstrate that loading, transformation, filtering, graph creation, and analysis can be performed in a single Flink program.
Overview
- What is Gelly?
- Graph Representation and Creation
- Transformations and Utilities
- Iterative Graph Processing
- Library of Graph Algorithms
- Use-Case: Music Profiles
- Ongoing and Future Work
What is Gelly? #
Gelly is a Graph API for Flink. It is currently supported in both Java and Scala. The Scala methods are implemented as wrappers on top of the basic Java operations. The API contains a set of utility functions for graph analysis, supports iterative graph processing and introduces a library of graph algorithms.
Graph Representation and Creation #
In Gelly, a graph is represented by a DataSet of vertices and a DataSet of edges.
A vertex is defined by its unique ID and a value, whereas an edge is defined by its source ID,
target ID, and value. A vertex or edge for which a value is not specified will simply have the
value type set to NullValue
.
A graph can be created from:
- DataSet of edges and an optional DataSet of vertices using
Graph.fromDataSet()
- DataSet of Tuple3 and an optional DataSet of Tuple2 using
Graph.fromTupleDataSet()
- Collection of edges and an optional Collection of vertices using
Graph.fromCollection()
In all three cases, if the vertices are not provided, Gelly will automatically produce the vertex IDs from the edge source and target IDs.
Transformations and Utilities #
These are methods of the Graph class and include common graph metrics, transformations and mutations as well as neighborhood aggregations.
Common Graph Metrics #
These methods can be used to retrieve several graph metrics and properties, such as the number of vertices, edges and the node degrees.
Transformations #
The transformation methods enable several Graph operations, using high-level functions similar to the ones provided by the batch processing API. These transformations can be applied one after the other, yielding a new Graph after each step, in a fashion similar to operators on DataSets:
inputGraph.getUndirected().mapEdges(new CustomEdgeMapper());
Transformations can be applied on:
- Vertices:
mapVertices
,joinWithVertices
,filterOnVertices
,addVertex
, … - Edges:
mapEdges
,filterOnEdges
,removeEdge
, … - Triplets (source vertex, target vertex, edge):
getTriplets
Neighborhood Aggregations #
Neighborhood methods allow vertices to perform an aggregation on their first-hop neighborhood. This provides a vertex-centric view, where each vertex can access its neighboring edges and neighbor values.
reduceOnEdges()
provides access to the neighboring edges of a vertex,
i.e. the edge value and the vertex ID of the edge endpoint. In order to also access the
neighboring vertices’ values, one should call the reduceOnNeighbors()
function.
The scope of the neighborhood is defined by the EdgeDirection parameter, which can be IN, OUT or ALL,
to gather in-coming, out-going or all edges (neighbors) of a vertex.
The two neighborhood
functions mentioned above can only be used when the aggregation function is associative and commutative.
In case the function does not comply with these restrictions or if it is desirable to return zero,
one or more values per vertex, the more general groupReduceOnEdges()
and
groupReduceOnNeighbors()
functions must be called.
Consider the following graph, for instance:
Assume you would want to compute the sum of the values of all incoming neighbors for each vertex.
We will call the reduceOnNeighbors()
aggregation method since the sum is an associative and commutative operation and the neighbors’ values are needed:
graph.reduceOnNeighbors(new SumValues(), EdgeDirection.IN);
The vertex with id 1 is the only node that has no incoming edges. The result is therefore:
Iterative Graph Processing #
During the past few years, many different programming models for distributed graph processing have been introduced: vertex-centric, partition-centric, gather-apply-scatter, edge-centric, neighborhood-centric. Each one of these models targets a specific class of graph applications and each corresponding system implementation optimizes the runtime respectively. In Gelly, we would like to exploit the flexible dataflow model and the efficient iterations of Flink, to support multiple distributed graph processing models on top of the same system.
Currently, Gelly has methods for writing vertex-centric programs and provides support for programs
implemented using the gather-sum(accumulate)-apply model. We are also considering to offer support
for the partition-centric computation model, using Fink’s mapPartition()
operator.
This model exposes the partition structure to the user and allows local graph structure exploitation
inside a partition to avoid unnecessary communication.
Vertex-centric #
Gelly wraps Flink’s Spargel APi to
support the vertex-centric, Pregel-like programming model. Gelly’s runVertexCentricIteration
method accepts two user-defined functions:
- MessagingFunction: defines what messages a vertex sends out for the next superstep.
- VertexUpdateFunction:* defines how a vertex will update its value based on the received messages.
The method will execute the vertex-centric iteration on the input Graph and return a new Graph, with updated vertex values.
Gelly’s vertex-centric programming model exploits Flink’s efficient delta iteration operators. Many iterative graph algorithms expose non-uniform behavior, where some vertices converge to their final value faster than others. In such cases, the number of vertices that need to be recomputed during an iteration decreases as the algorithm moves towards convergence.
For example, consider a Single Source Shortest Paths problem on the following graph, where S is the source node, i is the iteration counter and the edge values represent distances between nodes:
In each iteration, a vertex receives distances from its neighbors and adopts the minimum of these distances and its current distance as the new value. Then, it propagates its new value to its neighbors. If a vertex does not change value during an iteration, there is no need for it to propagate its old distance to its neighbors; as they have already taken it into account.
Flink’s IterateDelta
operator permits exploitation of this property as well as the
execution of computations solely on the active parts of the graph. The operator receives two inputs:
- the Solution Set, which represents the current state of the input and
- the Workset, which determines which parts of the graph will be recomputed in the next iteration.
In the SSSP example above, the Workset contains the vertices which update their distances. The user-defined iterative function is applied on these inputs to produce state updates. These updates are efficiently applied on the state, which is kept in memory.
Internally, a vertex-centric iteration is a Flink delta iteration, where the initial Solution Set is the vertex set of the input graph and the Workset is created by selecting the active vertices, i.e. the ones that updated their value in the previous iteration. The messaging and vertex-update functions are user-defined functions wrapped inside coGroup operators. In each superstep, the active vertices (Workset) are coGrouped with the edges to generate the neighborhoods for each vertex. The messaging function is then applied on each neighborhood. Next, the result of the messaging function is coGrouped with the current vertex values (Solution Set) and the user-defined vertex-update function is applied on the result. The output of this coGroup operator is finally used to update the Solution Set and create the Workset input for the next iteration.
Gather-Sum-Apply #
Gelly supports a variation of the popular Gather-Sum-Apply-Scatter computation model,
introduced by PowerGraph. In GSA, a vertex pulls information from its neighbors as opposed to the
vertex-centric approach where the updates are pushed from the incoming neighbors.
The runGatherSumApplyIteration()
accepts three user-defined functions:
- GatherFunction: gathers neighboring partial values along in-edges.
- SumFunction: accumulates/reduces the values into a single one.
- ApplyFunction: uses the result computed in the sum phase to update the current vertex’s value.
Similarly to vertex-centric, GSA leverages Flink’s delta iteration operators as, in many cases, vertex values do not need to be recomputed during an iteration.
Let us reconsider the Single Source Shortest Paths algorithm. In each iteration, a vertex:
- Gather retrieves distances from its neighbors summed up with the corresponding edge values;
- Sum compares the newly obtained distances in order to extract the minimum;
- Apply and finally adopts the minimum distance computed in the sum step, provided that it is lower than its current value. If a vertex’s value does not change during an iteration, it no longer propagates its distance.
Internally, a Gather-Sum-Apply Iteration is a Flink delta iteration where the initial solution set is the vertex input set and the workset is created by selecting the active vertices.
The three functions: gather, sum and apply are user-defined functions wrapped in map, reduce and join operators respectively. In each superstep, the active vertices are joined with the edges in order to create neighborhoods for each vertex. The gather function is then applied on the neighborhood values via a map function. Afterwards, the result is grouped by the vertex ID and reduced using the sum function. Finally, the outcome of the sum phase is joined with the current vertex values (solution set), the values are updated, thus creating a new workset that serves as input for the next iteration.
Library of Graph Algorithms #
We are building a library of graph algorithms in Gelly, to easily analyze large-scale graphs.
These algorithms extend the GraphAlgorithm
interface and can be simply executed on
the input graph by calling a run()
method.
We currently have implementations of the following algorithms:
- PageRank
- Single-Source-Shortest-Paths
- Label Propagation
- Community Detection (based on this paper)
- Connected Components
- GSA Connected Components
- GSA PageRank
- GSA Single-Source-Shortest-Paths
Gelly also offers implementations of common graph algorithms through examples. Among them, one can find graph weighting schemes, like Jaccard Similarity and Euclidean Distance Weighting, as well as computation of common graph metrics.
Use-Case: Music Profiles #
In the following section, we go through a use-case scenario that combines the Flink DataSet API with Gelly in order to process users’ music preferences to suggest additions to their playlist.
First, we read a user’s music profile which is in the form of user-id, song-id and the number of plays that each song has. We then filter out the list of songs the users do not wish to see in their playlist. Then we compute the top songs per user (i.e. the songs a user listened to the most). Finally, as a separate use-case on the same data set, we create a user-user similarity graph based on the common songs and use this resulting graph to detect communities by calling Gelly’s Label Propagation library method.
For running the example implementation, please use the 0.10-SNAPSHOT version of Flink as a dependency. The full example code base can be found here. The public data set used for testing can be found here. This data set contains 48,373,586 real user-id, song-id and play-count triplets.
Note: The code snippets in this post try to reduce verbosity by skipping type parameters of generic functions. Please have a look at the full example for the correct and complete code.
Filtering out Bad Records #
After reading the (user-id, song-id, play-count)
triplets from a CSV file and after parsing a
text file in order to retrieve the list of songs that a user would not want to include in a
playlist, we use a coGroup function to filter out the mismatches.
// read the user-song-play triplets.
DataSet<Tuple3<String, String, Integer>> triplets =
getUserSongTripletsData(env);
// read the mismatches dataset and extract the songIDs
DataSet<Tuple3<String, String, Integer>> validTriplets = triplets
.coGroup(mismatches).where(1).equalTo(0)
.with(new CoGroupFunction() {
void coGroup(Iterable triplets, Iterable invalidSongs, Collector out) {
if (!invalidSongs.iterator().hasNext()) {
for (Tuple3 triplet : triplets) { // valid triplet
out.collect(triplet);
}
}
}
}
The coGroup simply takes the triplets whose song-id (second field) matches the song-id from the mismatches list (first field) and if the iterator was empty for a certain triplet, meaning that there were no mismatches found, the triplet associated with that song is collected.
Compute the Top Songs per User #
As a next step, we would like to see which songs a user played more often. To this end, we build a user-song weighted, bipartite graph in which edge source vertices are users, edge target vertices are songs and where the weight represents the number of times the user listened to that certain song.
// create a user -> song weighted bipartite graph where the edge weights
// correspond to play counts
Graph<String, NullValue, Integer> userSongGraph = Graph.fromTupleDataSet(validTriplets, env);
Consult the Gelly guide for guidelines on how to create a graph from a given DataSet of edges or from a collection.
To retrieve the top songs per user, we call the groupReduceOnEdges function as it perform an aggregation over the first hop neighborhood taking just the edges into consideration. We will basically iterate through the edge value and collect the target (song) of the maximum weight edge.
//get the top track (most listened to) for each user
DataSet<Tuple2> usersWithTopTrack = userSongGraph
.groupReduceOnEdges(new GetTopSongPerUser(), EdgeDirection.OUT);
class GetTopSongPerUser implements EdgesFunctionWithVertexValue {
void iterateEdges(Vertex vertex, Iterable<Edge> edges) {
int maxPlaycount = 0;
String topSong = "";
for (Edge edge : edges) {
if (edge.getValue() > maxPlaycount) {
maxPlaycount = edge.getValue();
topSong = edge.getTarget();
}
}
return new Tuple2(vertex.getId(), topSong);
}
}
Creating a User-User Similarity Graph #
Clustering users based on common interests, in this case, common top songs, could prove to be very useful for advertisements or for recommending new musical compilations. In a user-user graph, two users who listen to the same song will simply be linked together through an edge as depicted in the figure below.
To form the user-user graph in Flink, we will simply take the edges from the user-song graph (left-hand side of the image), group them by song-id, and then add all the users (source vertex ids) to an ArrayList.
We then match users who listened to the same song two by two, creating a new edge to mark their common interest (right-hand side of the image).
Afterwards, we perform a distinct()
operation to avoid creation of duplicate data.
Considering that we now have the DataSet of edges which present interest, creating a graph is as
straightforward as a call to the Graph.fromDataSet()
method.
// create a user-user similarity graph:
// two users that listen to the same song are connected
DataSet<Edge> similarUsers = userSongGraph.getEdges()
// filter out user-song edges that are below the playcount threshold
.filter(new FilterFunction<Edge<String, Integer>>() {
public boolean filter(Edge<String, Integer> edge) {
return (edge.getValue() > playcountThreshold);
}
})
.groupBy(1)
.reduceGroup(new GroupReduceFunction() {
void reduce(Iterable<Edge> edges, Collector<Edge> out) {
List users = new ArrayList();
for (Edge edge : edges) {
users.add(edge.getSource());
for (int i = 0; i < users.size() - 1; i++) {
for (int j = i+1; j < users.size() - 1; j++) {
out.collect(new Edge(users.get(i), users.get(j)));
}
}
}
}
})
.distinct();
Graph similarUsersGraph = Graph.fromDataSet(similarUsers).getUndirected();
After having created a user-user graph, it would make sense to detect the various communities
formed. To do so, we first initialize each vertex with a numeric label using the
joinWithVertices()
function that takes a data set of Tuple2 as a parameter and joins
the id of a vertex with the first element of the tuple, afterwards applying a map function.
Finally, we call the run()
method with the LabelPropagation library method passed
as a parameter. In the end, the vertices will be updated to contain the most frequent label
among their neighbors.
// detect user communities using label propagation
// initialize each vertex with a unique numeric label
DataSet<Tuple2<String, Long>> idsWithInitialLabels = DataSetUtils
.zipWithUniqueId(similarUsersGraph.getVertexIds())
.map(new MapFunction<Tuple2<Long, String>, Tuple2<String, Long>>() {
@Override
public Tuple2<String, Long> map(Tuple2<Long, String> tuple2) throws Exception {
return new Tuple2<String, Long>(tuple2.f1, tuple2.f0);
}
});
// update the vertex values and run the label propagation algorithm
DataSet<Vertex> verticesWithCommunity = similarUsersGraph
.joinWithVertices(idsWithlLabels, new MapFunction() {
public Long map(Tuple2 idWithLabel) {
return idWithLabel.f1;
}
})
.run(new LabelPropagation(numIterations))
.getVertices();
Ongoing and Future Work #
Currently, Gelly matches the basic functionalities provided by most state-of-the-art graph processing systems. Our vision is to turn Gelly into more than “yet another library for running PageRank-like algorithms” by supporting generic iterations, implementing graph partitioning, providing bipartite graph support and by offering numerous other features.
We are also enriching Flink Gelly with a set of operators suitable for highly skewed graphs as well as a Graph API built on Flink Streaming.
In the near future, we would like to see how Gelly can be integrated with graph visualization tools, graph database systems and sampling techniques.
Curious? Read more about our plans for Gelly in the roadmap.


Full
Longer parameter versions of System.Linq.ParallelEnumerable
methods
Functions and values
| Function or value | Description |
aggregate (...)
Signature: seedFactory:(unit -> 'TAccumulate) -> updateAccumulatorFunc:('TAccumulate -> 'TSource -> 'TAccumulate) -> combineAccumulatorsFunc:('TAccumulate -> 'TAccumulate -> 'TAccumulate) -> resultSelector:('TAccumulate -> 'TResult) -> source:(type) -> 'TResult
Type parameters: 'TAccumulate, 'TSource, 'TResult |
Go to GitHub source |
any predicate source
Signature: predicate:('TSource -> (type)) -> source:(type) -> bool
Type parameters: 'TSource |
Calls |
asOrdered source
Signature: source:(type) -> (type)
|
Calls |
contains value comparer source
Signature: value:'?7999 -> comparer:(type) -> source:(type) -> bool
Type parameters: '?7999 |
Calls |
count predicate source
Signature: predicate:('TSource -> (type)) -> source:(type) -> int
Type parameters: 'TSource |
Calls |
defaultIfEmpty defaultValue source
Signature: defaultValue:'?8003 -> source:(type) -> (type)
Type parameters: '?8003 |
Calls |
distinct comparer source
Signature: comparer:(type) -> source:(type) -> (type)
Type parameters: '?8005 |
Calls |
except comparer first second
Signature: comparer:(type) -> first:(type) -> second:(type) -> (type)
Type parameters: 'TSource |
Calls |
first predicate source
Signature: predicate:('TSource -> (type)) -> source:(type) -> 'TSource
Type parameters: 'TSource |
Calls |
firstOrDefault predicate source
Signature: predicate:('TSource -> (type)) -> source:(type) -> 'TSource
Type parameters: 'TSource |
Calls |
groupBy (...)
Signature: keySelector:('TSource -> 'TKey) -> elementSelector:('TSource -> 'TElement) -> resultSelector:('TKey -> (type) -> 'TResult) -> comparer:(type) -> source:(type) -> (type)
Type parameters: 'TSource, 'TKey, 'TElement, 'TResult |
Go to GitHub source |
groupJoin (...)
Signature: outerKeySelector:('TOuter -> 'TKey) -> innerKeySelector:('TInner -> 'TKey) -> resultSelector:('TOuter -> (type) -> 'TResult) -> comparer:(type) -> outer:(type) -> inner:(type) -> (type)
Type parameters: 'TOuter, 'TKey, 'TInner, 'TResult |
Go to GitHub source |
intersect comparer first second
Signature: comparer:(type) -> first:(type) -> second:(type) -> (type)
Type parameters: 'TSource |
Calls |
join (...)
Signature: outerKeySelector:('TOuter -> 'TKey) -> innerKeySelector:('TInner -> 'TKey) -> resultSelector:('TOuter -> 'TInner -> 'TResult) -> comparer:(type) -> outer:(type) -> inner:(type) -> (type)
Type parameters: 'TOuter, 'TKey, 'TInner, 'TResult |
Go to GitHub source |
last predicate source
Signature: predicate:('TSource -> (type)) -> source:(type) -> 'TSource
Type parameters: 'TSource |
Calls |
lastOrDefault predicate source
Signature: predicate:('TSource -> (type)) -> source:(type) -> 'TSource
Type parameters: 'TSource |
Calls |
longCount predicate source
Signature: predicate:('TSource -> (type)) -> source:(type) -> int64
Type parameters: 'TSource |
Calls |
orderBy keySelector comparer source
Signature: keySelector:('TSource -> 'TKey) -> comparer:(type) -> source:(type) -> (type)
Type parameters: 'TSource, 'TKey |
Calls |
orderByDescending (...)
Signature: keySelector:('TSource -> 'TKey) -> comparer:(type) -> source:(type) -> (type)
Type parameters: 'TSource, 'TKey |
Calls |
select selector source
Signature: selector:('TSource -> (type) -> 'TResult) -> source:(type) -> (type)
Type parameters: 'TSource, 'TResult |
Calls |
selectMany (...)
Signature: collectionSelector:('TSource -> (type) -> (type)) -> resultSelector:('TSource -> 'TCollection -> 'TResult) -> source:(type) -> (type)
Type parameters: 'TSource, 'TCollection, 'TResult |
Go to GitHub source |
sequenceEqual comparer first second
Signature: comparer:(type) -> first:(type) -> second:(type) -> bool
Type parameters: 'TSource |
Calls |
single predicate source
Signature: predicate:('TSource -> (type)) -> source:(type) -> 'TSource
Type parameters: 'TSource |
Calls |
singleOrDefault predicate source
Signature: predicate:('TSource -> (type)) -> source:(type) -> 'TSource
Type parameters: 'TSource |
Calls |
skipWhile predicate source
Signature: predicate:('TSource -> (type) -> (type)) -> source:(type) -> (type)
Type parameters: 'TSource |
Calls |
takeWhile predicate source
Signature: predicate:('TSource -> (type) -> (type)) -> source:(type) -> (type)
Type parameters: 'TSource |
Calls |
thenBy keySelector comparer source
Signature: keySelector:('TSource -> 'TKey) -> comparer:(type) -> source:(type) -> (type)
Type parameters: 'TSource, 'TKey |
Calls |
thenByDescending (...)
Signature: keySelector:('TSource -> 'TKey) -> comparer:(type) -> source:(type) -> (type)
Type parameters: 'TSource, 'TKey |
Calls |
toDictionary (...)
Signature: keySelector:('TSource -> 'TKey) -> elementSelector:('TSource -> 'TElement) -> comparer:(type) -> source:(type) -> (type)
Type parameters: 'TSource, 'TKey, 'TElement |
Go to GitHub source |
toLookup (...)
Signature: keySelector:('TSource -> 'TKey) -> elementSelector:('TSource -> 'TElement) -> comparer:(type) -> source:(type) -> (type)
Type parameters: 'TSource, 'TKey, 'TElement |
Go to GitHub source |
union comparer first second
Signature: comparer:(type) -> first:(type) -> second:(type) -> (type)
Type parameters: 'TSource |
Calls |
where predicate source
Signature: predicate:('TSource -> (type) -> (type)) -> source:(type) -> (type)
Type parameters: 'TSource |
Calls |


Movie Recommendation System
This is not actual implementation, it is just a demonstration :)
Choose Parameter...
Title
Genre
Cast
Director
Get My Movies!
Poster
Title
Genres
Cast
Director


Instructions:
ESC: reset
Double click a service for details
mouse wheel = zoom
left mouse = pan
right mouse = rotate


Stories from Around the World
by FCIT
The Rat Princess
by Frank Rinder
Additional Information
- Year Published: 1918
- Language: English
- Country of Origin: Japan
- Source: Sara Cone Bryant, ed., How to Tell Stories to Children, and Some Stories to Tell
-
Readability:
- Flesch–Kincaid Level: 5.1
- Word Count: 743
- Genre: Fairy Tale/Folk Tale
- Keywords: fate
- ✎ Cite This
Downloads
Once upon a time, there was a Rat Princess, who lived with her father, the Rat King, and her mother, the Rat Queen, in a ricefield in far away Japan.  When she grew up, they would not let any of the rat princes come to visit her, and they decided at last that no one should marry her till they had found the most powerful person in the whole world; no one else was good enough. And the Father Rat started out to find the most powerful person in the whole world. The wisest and oldest rat in the ricefield said that the Sun must be the most powerful person, because he made the rice grow and ripen; so the Rat King went to find the Sun. He climbed up the highest mountain, ran up the path of a rainbow, and travelled and travelled across the sky till he came to the Sun’s house.
“What do you want, little brother?” the Sun said, when he saw him.
“I come,” said the Rat King, very importantly, “to offer you the hand of my daughter, the princess, because you are the most powerful person in the world; no one else is good enough.”
“Ha, ha!” laughed the jolly round Sun, and winked with his eye. “You are very kind, little brother, but if that is the case the princess is not for me; the Cloud is more powerful than I am; when he passes over me I cannot shine.”
“Oh, indeed,” said the Rat King, “then you are not my man at all”; and he left the Sun without more words. The Sun laughed and winked to himself. And the Rat King travelled and travelled across the sky till he came to the Cloud’s house.
“What do you want, little brother?” sighed the Cloud when he saw him.
“I come to offer you the hand of my daughter, the princess,” said the Rat King, “because you are the most powerful person in the world; the Sun said so, and no one else is good enough.”
The Cloud sighed again. “I am not the most powerful person,” he said; “the Wind is stronger than I,—when he blows, I have to go wherever he sends me.”
“Then you are not the person for my daughter,” said the Rat King proudly; and he started at once to find the Wind. He travelled and travelled across the sky, till he came at last to the Wind’s house, at the very edge of the world.
When the Wind saw him coming he laughed a big, gusty laugh, “Ho, ho!” and asked him what he wanted; and when the Rat King told him that he had come to offer him the Rat Princess’s hand because he was the most powerful person in the world, the Wind shouted a great gusty shout, and said, “No, no, I am not the strongest; the Wall that man has made is stronger than I; I cannot make him move, with all my blowing; go to the Wall, little brother!”
And the Rat King climbed down the sky-path again, and travelled and travelled across the earth till he came to the Wall. It was quite near his own ricefield.
“What do you want, little brother?” grumbled the Wall when he saw him.
“I come to offer you the hand of the princess, my daughter, because you are the most powerful person in the world, and no one else is good enough.”
“Ugh, ugh,” grumbled the Wall, “I am not the strongest; the big grey Rat who lives in the cellar is stronger than I. When he gnaws and gnaws at me I crumble and crumble, and at last I fall; go to the Rat, little brother.”
And so, after going all over the world to find the strongest person, the Rat King had to marry his daughter to a rat, after all; but the princess was very glad of it, for she wanted to marry the grey Rat, all the time.


Assessment Policy
Definition: The term “Assessment” refers generally to the gathering and interpretation of information related to a student’s learning abilities, learning attainments, learning strengths and learning needs.
Rationale
The Education Act (1998) requires schools to:
Assess the effectiveness of the teaching and learning process
Regularly evaluate students
Periodically report the results of the evaluation to students and their parents
In fulfilling this requirement, schools develop assessment procedures, which provide an accurate account of a student’s progress and achievement.
In Mount Sion CBS Secondary School, as well as focusing on material relevant to terminal examinations, teachers engage in a broad range of teaching strategies and methodologies which provide our students with a deep appreciation of their respective subject areas.
Aims of Assessment
Students: Assessment should enable students to:
Improve their achievement levels
Direct their own learning
Become motivated to achieve their full potential
Receive feedback in a timely and constructive manner
Teachers: Assessment should enable teachers to:
Assess progress of students and thereby adapt teaching plans as required
Evaluate their students’ learning needs
Establish baseline data in relation to a student’s attainment in subjects
Ascertain whether it is appropriate for a student to continue studying in any given subject area
Identify appropriate levels of study - Higher, Ordinary or Foundation – for students
Improve students’ ability to improve the standard of their work
Assist in the identification of students who may need to be referred for additional learning support
Assess a student’s eligibility for additional services and to inform consultations with NEPS or private psychologist where necessary.
Provide the students and parents with information regarding progress
Evaluate teaching and learning styles and outcomes
Parents: Reporting of assessment should enable parents to:
Become actively involved in their son’s learning
Understand and encourage their son’s progress in the different subjects
Use the feedback to motivate and guide their son through school years
Policy
It is school policy at Mount Sion CBS Secondary School to encourage and support ongoing assessment as part of the teaching and learning process. In this regard, there will be a clear calendar of reporting students’ progress to parents/guardians. As part of ongoing assessment, teachers are expected to keep an account of the progress of students in the variety of tasks which may occur in the teaching/learning process.
Policy Objectives: The primary objectives of the policy are to:
Facilitate improved teaching and pupil learning
Create a consistent and standardised assessment procedure for monitoring and reporting achievement in the school
Co-ordinate assessment procedures within departments on a whole school basis
Ensure that each student realises his/her full academic potential in a learning environment
Forms of Assessment
1. Informal Assessment
The most common forms of assessment used in the school are of an informal nature, and these take place regularly in the classroom. These informal assessments are at the discretion of individual teachers who will use them constructively for learning to take place. Timely feedback and constructive advice are vital components in this process. Where appropriate, teachers will keep records of these assessments to provide an insight into a student’s progress over time and to fully inform the learning process. Informal assessment methods include:
Worksheets & written classwork
Question and answer sessions in class
Essays & assignments
Reading & writing in class
Sample examination questions
Homework – written or learned
Revision sheets
Show Me Boards
Spelling Tests & Listening Comprehension
‘Pass the Ball’ question games
2. Formal Assessment
There are many types of formal assessment that take place in the school environment:
Monthly/End of Topic Test – These forms of assessment will be conducted at the discretion of the teacher. They are presented on a monthly, end of chapter or end of topic basis. These formal tests are a vital component in providing feedback to teachers, students and parents on the level of each student’s attainment and learning in that specific subject area. Constructive feedback will be given by teachers to students who partake in these tests, and the results of these tests will be used in the school’s formal reporting structure where appropriate.
House Exams
1st, 2nd and 5th Years: 1st year students will have exams pre mid-term in October and a mark only report without comments will be sent to parents. Students will sit their Christmas Exams in regular class time in early December. The Summer Exams will take place at the end of May with reports and comments sent to parents. The duration of these exams may vary from subject to subject. The purpose of these exams is to (i) measure student learning and attainment in each subject area and (ii) give students experience in sitting formal exams similar in nature to the state examinations.
All house exams will follow the same operating rules as state exams. The results of these Christmas Exams and Summer Exams will be reported in the Christmas Reports and Summer Reports respectively.
3rd Year and 6th Years: Students will sit exams in regular class time at Christmas. These class exams will be based on comprehensive Junior Cert and Leaving Cert course content, covered to date in each subject area. 3rd and 6th Christmas Reports will include the results of these Christmas Exams.
3rd and 6th Year students will undertake Pre Junior and Pre Leaving Cert Mock Exams in February each year to prepare them for their state examinations in June. Where Psychological Assessments recommend that students receive Reasonable Accommodations in State Examinations, Mount Sion CBS will make every effort possible to extend this facility to those students in Pre Exams. This will be determined by the financial and personnel resources available to the school at that particular time. These mock exams are seen as being very beneficial to the students to enable them to (i) measure their progress in their Junior Cert and Leaving Cert years and (ii) gain valuable experience in time management and answering techniques.
The results of these mock exams will form the basis for the Pre Junior Cert and Pre Leaving Cert Reports sent out by the school. In cases where there have been marked under-performances by individual students, the Career Guidance and/or the Deputy Principal/ Principal may engage with the student in relation to their attitude to their studies and encourage improvement.
Since external assessment for both Junior and Leaving Certificate begins in April no further school based formal assessment takes place. However the teacher’s own assessment record is of vital importance in guiding the students’ preparation. Students in 3rd Year and 6th Year do not sit any formal house exams at the end of May and do not receive Summer Reports.
Transition Year: Students do not sit any formal state or house exams in the school. Their Christmas and Summer Reports are based on continuous assessments in the classroom as determined by their individual subject teacher. As Transition Year is focused on experiential learning and personal development, the forms of assessment used may vary from the standard practices used in other years. Sources of assessment and affirmation in Transition Year include participation in special projects and reports from work experience
State Exams: The Junior and Leaving Certificate Examinations will be conducted in early June by the State Examinations Commission under the aegis of the Department of Education & Skills. The school will strive to ensure that all 3rd and 6th Year students participate - as directed by the State Exams Commission - in the many assessments that make up the Junior Cert and Leaving Cert exams. These assessments which may include orals, course work, practical work, portfolio work amongst others, will be operated in accordance with the regulations set down by the State Examinations Commission.
Incoming First Years: All incoming first years undergo formal assessment of verbal and numerical proficiency during February preceding initial enrolment. Using established assessment instruments, school personnel identify students from this cohort who may lag behind their peer group and make recommendations to the Principal in relation to the provision of Learning Support. The findings from these assessments are strictly confidential and are used for no purpose other than the identification of learning challenges. All classes in first year are formed on a mixed ability and random basis.
General Guidelines: The form that these informal and formal assessments will take, are dependent on the subject area being assessed and will vary from subject to subject. Certain forms of assessment may oftentimes be peculiar to a particular subject. All forms of assessment above will strive to meet the additional needs of students in the classroom when they arise. Classroom strategies such as differentiated assessment, visual aids, hearing aids, spelling waivers, increased time and dictionaries amongst others, may be used from time to time, as determined by the individual subject teacher.
Students and parents have little opportunity to assess students’ academic progress against an established benchmark until the Junior Certificate at the end of Third Year. Where feasible, and as determined by each department plan, common assessments across a year group will be used in each subject area. This will provide for increased cooperation and a common benchmark across each subject area. Having clearly defined targets for each year would greatly facilitate students changing class and teachers taking up a class in mid-cycle.
Students are expected to treat all of the above forms of assessment with respect. Any student who willingly fails to participate fully, or who acts in a disrespectful way towards any form of assessment in the school will be subject to school disciplinary procedures.
Assessment for Learning
Assessment should support learning as well as measure the outcomes. Effective assessment for learning enables pupils to understand how to improve their work by:
Helping them to understand the assessment criteria before an assignment is begun
Telling them what they have done well and what they could improve on
Telling them how to improve their work
Making available examples of work which do meet the criteria so they can see how to improve their own work
The Conduct of Examinations:
In order that school examinations (a) have a validity and integrity and (b) that they serve as a preparation for other more formal occasions, it is essential that in-school examinations be well administered.
To this end:
Proper answer booklets will be provided
Papers should be properly presented, clearly laid out and well copied
The rules for each examination event and each examination centre will be clearly stated and full compliance expected. This will normally be done by the House Examinations Co-ordinator
Supervisors should be diligent in their duties
Students may not leave the exam centre before 2/3 of duration of the examination.
Where a student violates the rules and protocols for an examination, they will be cautioned by the supervising teacher in relation to any further violation. The student may be removed to an alternative venue.
Assessment and Reporting
There are different methods whereby the results of formal and informal assessments will be reported to parents.
Student Journal: Students will record all continuous assessment results in their journals for all subjects. The teacher may communicate the outcome of any assessment to the parents/guardian of a student by writing a note in the student’s journal. The parent/guardian is expected to acknowledge receipt of this note by countersigning it.
Direct Contact: The teacher may decide as they see fit to contact the parent/guardian directly with respect to the outcome of any assessment undertaken by a student
Signature of Parent on Assessment: The teacher may send the corrected assessment itself home with the student to be viewed and signed by the parent/guardian.
Parent Teacher Meetings: There are Parent Teacher Meetings for all Year Groups throughout the academic year. At these meetings it will be possible for teachers to relay the outcomes of varying assessments to parents/guardians of individual students to generate an accurate picture of the student’s progress.
Written reports: Formal written reports are posted to the parents/guardians of students on two occasions during the academic year, depending which year group they are in. 1st, 2nd, Transition Year and 5th Year parents/guardians will receive their Christmas Report in December before the Christmas holidays, and their Summer Report during the summer holidays. Parents/guardians of 3rd and 6th will also receive a Christmas Report in December, but will then receive another report in April with the results of their Pre Junior and Pre Leaving Cert Exams. 3rd and 6th Year parents/guardians do not receive a Summer Report.
This policy has been ratified by the Board of Management