Row's and columns are what it's all about on a webpage.It's straight forward cartesian axis of 12 vertical divisions and any number of horizontal ones that define where on a page different elements are placed.
Increasingly websites are being viewed on a variety of devices.Not just the desktop of yesteryear but tablets,phlablets and of course the ubiquitous smartphone. Web design that worked so well on a full screen was soon recognised to be less than ideal on one a fraction of the size. Rather than design multiple sites, methods are now implimented whereby a sites layout and features automatically modify and compensate for different viewing portal options.
In order for a web browser to interpret the instructions and display the elements of a pages design, certain nomenclatureal forms must be observed. For reasons of brevity and clariety it is not correct to simply designate a phrase 'Big' then assign it a large font size. One would instead for example utilize an h1 tag, recognisable by a browser as a heading of a certain size.
The facinating challanges that arise whilst attempting to replicate something that has worked previously or has been seen to work in other cases. A f*$?%!# box for example, yes that one to the left, that inexplicably and infuriatingly demonstrate aberant behaviour.


--- layout: post title: Everything, in no particular order date: 2013-04-01 21:49:05.000000000 -07:00 type: post published: true status: publish categories: - 30 Days in April tags: - 30 Days in April - Android - Depreciation Guild - Digital audio - radiohead - Spotify meta: _publicize_pending: '1' _edit_last: '6192' original_post_id: '451' _wp_old_slug: '451' author: login: falsefridays email<EMAIL_ADDRESS>display_name: Jim Withington first_name: '' last_name: '' excerpt: Spotify, GBV, and the consumption of music. ---
I’m an avid Spotify listener. I have told people, multiple times, that if needed to cut back on my subscriptions, Spotify would be the last to go (sorry, Erin). It really has changed the way that I listen to music.
In the past, if I wanted to explore something random—say, for instance, a jazzy track that I heard on the NBA 2k13 soundtrack—I could do a few things:
And so I ended up either getting things in a sketchy way or, basically, forgetting about it.
Now, I can just find it on Spotify. This was possible with Rhapsody, but Spotify’s professed love for amping up their servers so that it seems like the files are all local really works.
Over the course of this site’s existence, I wrote a draft of a post that talked about bands giving music away (a la Radiohead and the The Depreciation Guild ), and I think that a looong time ago I wrote about how my musical taste was, like many of us, shifting to the single, because of my Kubrickian iPod.
Well, Spotify changed the game again for me, and maybe this is another snapshot of that. I have enough gadgets—and Android is popular enough now—that I can pretty much listen not to just to my music, but all of the music I can think of all the time (as long as I have a connection)(pro-tip: as long as you plan ahead, you can also download songs from Spotify. No connection needed!).
Oddly, this is leading to me moving more towards built in Spotify apps (to help me discover bands) or—get this—album listens.
As someone with limited exposure to GBV, can't decide if Spotify is made for a band like them, or if getting the actual release is crucial.
— J Wintr'y (@jwithy) April 2, 2013
I still don’t know the answer to my Guided By Voices question. Maybe the act of flipping the record, instead of searching in a text box, really does get me to be able to get into GBV more.
Or maybe I would never have bothered if I hadn’t decided to pay $10/month for all of the things.


I'm a Portuguese multidisciplinary designer currently living in Coimbra, Portugal.
I'm an open-minded and collaborative designer, who's passionate about helping growing startups by creating beautiful experiences that 'wow’ customers and increase business results.


| Author | Matthew Schinkel - borntechi.com, copyright © 2009, all rights reserved. |
| Adapted-by | |
| Compiler | 2.4o |
SPI Master software control. Routines for sending and receiving through SPI software in master mode
Jallib spi_master_hw library - Written by William Welch http://www.justanotherlanguage.org/content/jallib/tutorials/tutorial_spi_introduction http://ww1.microchip.com/downloads/en/devicedoc/spi.pdf
alias spi_master_sw_sdi is pin_sdi alias spi_master_sw_sdi_direction is pin_sdi_direction alias spi_master_sw_sdo is pin_sdo alias spi_master_sw_sdo_direction is pin_sdo_direction alias spi_master_sw_sck is pin_sck alias spi_master_sw_sck_direction is pin_sck_direction define spi inputs/outputs spi_master_sw_sdi_direction = input -- spi input spi_master_sw_sdo_direction = output -- spi output spi_master_sw_sck_direction = output -- spi clock include spi_master_sw spi_init(SPI_MODE_11) -- init spi, choose mode
spi_master_sw_init(byte in spi_mode)
spi_master_sw_set_speed(byte in spi_rate)
spi_master_sw_set_mode_11()
spi_master_sw_set_mode_00()
spi_master_sw_set_mode(byte in spi_mode)
spi_master_sw'put(byte in data)
spi_master_sw_set_mode_10()
spi_master_sw_set_mode_01()
var byte _spi_mode
No documentation found
var byte _spi_rate
No documentation found
spi_master_sw_init(byte in spi_mode)
init spi
spi_master_sw_set_speed(byte in spi_rate)
set spi speed (not actually implemented yet, just here for compatibility)
spi_master_sw_set_mode_11()
No documentation found
spi_master_sw_set_mode_00()
No documentation found
spi_master_sw_set_mode(byte in spi_mode)
set spi mode
spi_master_sw'put(byte in data)
put one byte of data onto the spi bus (pseudo variable)
spi_master_sw_set_mode_10()
No documentation found
spi_master_sw_set_mode_01()
No documentation found
spi_master_sw_exchange(byte in data) return byte
exchange data on the spi bus.
spi_master_sw'get() return byte
read one byte of data from the spi bus (pseudo variable)
| 18f27j53 | 18f27j53_mcp4922.jal |
| 18f46k22 | 18f46k22_glcd_touch_stm032qvt_003.jal |


Untitled Document
The following text is the License for this software. You should
find it identical to that contained in the file LICENSE in the
source distribution.
@bf{------------------ START OF THE LICENSE ------------------}
This program, bzip2
,
and associated library libbzip2
, are
Copyright (C) 1996-2002 Julian R Seward. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
-
Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
-
The origin of this software must not be misrepresented; you must
not claim that you wrote the original software. If you use this
software in a product, an acknowledgment in the product
documentation would be appreciated but is not required.
-
Altered source versions must be plainly marked as such, and must
not be misrepresented as being the original software.
-
The name of the author may not be used to endorse or promote
products derived from this software without specific prior written
permission.

Julian Seward, Cambridge, UK.
<EMAIL_ADDRESS>bzip2
/libbzip2
version 1.0.2 of 30 December 2001.
@bf{------------------ END OF THE LICENSE ------------------}
Web sites:
http://sources.redhat.com/bzip2
http://www.cacheprof.org
PATENTS: To the best of my knowledge, bzip2
does not use any patented
algorithms. However, I do not have the resources available to carry out
a full patent search. Therefore I cannot give any guarantee of the
above statement.
This document was generated
by Julian Seward on January, 5 2002
using texi2html


Syrian military attack causes massive blast in southern Idlib: video
by News DeskBEIRUT, LEBANON (12:00 P.M.) – The official Facebook page of the 25th Special Mission Forces reported that their troops (formerly Tiger Forces) unleashed a massive attack on the southern countryside of the Idlib Governorate this week, as they attempt to advance further into this region.
In a video released on Monday morning, the 25th Special Missions Forces can be seen striking the jihadist positions in southern Idlib.


Home
About
Products
Contact Us
I would love to hear from you.
Mobile phone:
0783430951
Email:
<EMAIL_ADDRESS>Address:
Kk 74 st
Name
Email
Subject
Message


Module: Sprockets::Utils
| Relationships & Source Files | |
| Namespace Children | |
|
Classes:
| |
| Extension / Inclusion / Inheritance Descendants | |
|
Extended In:
| |
|
Included In:
| |
| Defined in: | lib/sprockets/utils.rb, lib/sprockets/utils/gzip.rb |
Overview
Internal: Utils
, we didn’t know where else to put it! Functions may eventually be shuffled into more specific drawers.
Constant Summary
-
MODULE_INCLUDE_MUTEX =
private
# File 'lib/sprockets/utils.rb', line 121Mutex.new
-
WHITESPACE_ORDINALS =
private
# File 'lib/sprockets/utils.rb', line 71{0x0A => "\n", 0x20 => " ", 0x09 => "\t"}
Instance Method Summary
-
#concat_javascript_sources(buf, source)
Internal: Accumulate asset source to buffer and append a trailing semicolon if necessary.
-
#dfs(initial)
Internal: Post-order Depth-First search algorithm.
-
#dfs_paths(path)
Internal: Post-order Depth-First search algorithm that gathers all paths along the way.
-
#duplicable?(obj) ⇒ Boolean
Internal: Check if object can safely be .dup’d.
-
#hash_reassoc(hash, key_a, key_b = nil, &block)
Internal: Duplicate and store key/value on new frozen hash.
-
#hash_reassoc1(hash, key)
Internal: Duplicate and store key/value on new frozen hash.
-
#module_include(base, mod)
Internal: Inject into target module for the duration of the block.
-
#string_end_with_semicolon?(str) ⇒ Boolean
Internal: Check if string has a trailing semicolon.
Instance Method Details
#concat_javascript_sources(buf, source)
Internal: Accumulate asset source to buffer and append a trailing semicolon if necessary.
buf - String buffer to append to source - String source to append
Returns buf String.
# File 'lib/sprockets/utils.rb', line 100
def concat_javascript_sources(buf, source) return buf if source.bytesize <= 0 buf << source # If the source contains non-ASCII characters, indexing on it becomes O(N). # This will lead to O(N^2) performance in string_end_with_semicolon?, so we should use 32 bit encoding to make sure indexing stays O(1) source = source.encode(Encoding::UTF_32LE) unless source.ascii_only? return buf if string_end_with_semicolon?(source) # If the last character in the string was whitespace, # such as a newline, then we want to put the semicolon # before the whitespace. Otherwise append a semicolon. if whitespace = WHITESPACE_ORDINALS[source[-1].ord] buf[-1] = ";#{whitespace}" else buf << ";" end buf end
#dfs(initial)
Internal: Post-order Depth-First search algorithm.
Used for resolving asset dependencies.
initial - Initial Array of nodes to traverse. block -
node - Current node to get children of
Returns a Set of nodes.
# File 'lib/sprockets/utils.rb', line 165
def dfs(initial) nodes, seen = Set.new, Set.new stack = Array(initial).reverse while node = stack.pop if seen.include?(node) nodes.add(node) else seen.add(node) stack.push(node) stack.concat(Array(yield node).reverse) end end nodes end
#dfs_paths(path)
Internal: Post-order Depth-First search algorithm that gathers all paths along the way.
TODO: Rename function.
path - Initial Array node path block -
node - Current node to get children of
Returns an Array of node Arrays.
# File 'lib/sprockets/utils.rb', line 192
def dfs_paths(path) paths = [] stack = [path] seen = Set.new while path = stack.pop seen.add(path.last) paths << path children = yield path.last children.reverse_each do |node| stack.push(path + [node]) unless seen.include?(node) end end paths end
#duplicable?(obj) ⇒ Boolean
Internal: Check if object can safely be .dup’d.
Similar to ActiveSupport #duplicable?
check.
obj - Any Object
Returns false if .dup
would raise a TypeError, otherwise true.
# File 'lib/sprockets/utils.rb', line 17
def duplicable?(obj) case obj when NilClass, FalseClass, TrueClass, Symbol, Numeric false else true end end
#hash_reassoc(hash, key_a, key_b = nil, &block)
Internal: Duplicate and store key/value on new frozen hash.
Similar to Hash#store
for nested frozen hashes.
hash - Hash key_a - Object key. Use multiple keys for nested hashes. key_b - Object key. Use multiple keys for nested hashes. block - Receives current value at key.
Examples
config = {paths: ["/bin", "/sbin"]}.freeze
new_config = hash_reassoc(config, :paths) do |paths|
paths << "/usr/local/bin"
end
Returns duplicated frozen Hash.
# File 'lib/sprockets/utils.rb', line 61
def hash_reassoc(hash, key_a, key_b = nil, &block) if key_b hash_reassoc1(hash, key_a) do |value| hash_reassoc(value, key_b, &block) end else hash_reassoc1(hash, key_a, &block) end end
#hash_reassoc1(hash, key)
Internal: Duplicate and store key/value on new frozen hash.
Separated for recursive calls, always use hash_reassoc(hash, *keys).
hash - Hash key - Object key
Returns Hash.
# File 'lib/sprockets/utils.rb', line 34
def hash_reassoc1(hash, key) hash = hash.dup if hash.frozen? old_value = hash[key] old_value = old_value.dup if duplicable?(old_value) new_value = yield old_value new_value.freeze if duplicable?(new_value) hash.store(key, new_value) hash.freeze end
#module_include(base, mod)
Internal: Inject into target module for the duration of the block.
mod - Module
Returns result of block.
# File 'lib/sprockets/utils.rb', line 129
def module_include(base, mod) MODULE_INCLUDE_MUTEX.synchronize do old_methods = {} mod.instance_methods.each do |sym| old_methods[sym] = base.instance_method(sym) if base.method_defined?(sym) end mod.instance_methods.each do |sym| method = mod.instance_method(sym) if base.method_defined?(sym) base.send(:alias_method, sym, sym) end base.send(:define_method, sym, method) end yield ensure mod.instance_methods.each do |sym| base.send(:undef_method, sym) if base.method_defined?(sym) end old_methods.each do |sym, method| base.send(:define_method, sym, method) end end end
#string_end_with_semicolon?(str) ⇒ Boolean
Internal: Check if string has a trailing semicolon.
str - String
Returns true or false.
# File 'lib/sprockets/utils.rb', line 79
def string_end_with_semicolon?(str) i = str.size - 1 while i >= 0 c = str[i].ord i -= 1 next if WHITESPACE_ORDINALS[c] return c === 0x3B end true end


[
next
] [
prev
] [
prev-tail
] [
tail
] [
up
]
10
Magnetic coordinates (
ψ,𝜃,ζ
) with general toroidal angle
ζ
10.1
General toroidal angle
ζ
10.2
Contravariant form of magnetic ﬁeld in (
ψ,𝜃,ζ
) coordinates
10.3
Relation between the partial derivatives in (
ψ,𝜃,ϕ
) and (
ψ,𝜃,ζ
) coordinates
10.4
Steps to construct a straight-line magnetic coordinate system
10.5
Form of operator
B
⋅∇
in (
ψ,𝜃,ζ
) coordinates
10.6
Resonant surface of a perturbation
10.7
Helical angle used in tearing mode theory
10.8
Covariant form of magnetic ﬁeld in (
ψ,𝜃,ζ
) coordinate system
10.9
Form of operator (
B
×∇
ψ∕B
2
)
⋅∇
in (
ψ,𝜃,ζ
) coordinates
10.10
Radial diﬀerential operator
[
next
] [
prev
] [
prev-tail
] [
front
] [
up
]


Coder - Game Designer - Dad
I was born and raised in the beautiful Italian Riviera, near the France border, between Nice and Genova. My passion in the field of computer science dates back to the days when my parents bought me my first computer, a 486 with the DOS operating system and a 56k modem, while I was in 5th grade. Computers have always been my safe harbor, my constant companions together with the Internet.
I worked as SysAdmin for most of my carrer. I moved to the United States in 2015, precisely Boston MA, and I lived and worked there for 5 years. I recently moved back home in Italy and I started a new career as a software developer and project manager for a local business. As a long time gamer, It's always been a secret dream of mine to create my own indie video game and I'm working hard to achieve this goal in my spare time.
Email<EMAIL_ADDRESS>"And from my pillow, looking forth by light
of moon or favouring stars, I could behold
the antechapel where the statue stood
of Newton with his prism and silent face,
the marble index of a mind forever voyaging
through strange seas of thought, alone."
(William Wordsworth)


Test Personas are a way of asserting expected behaviour of
When you make a persona, you give it standard claims and visas that make it simulate a category of user
in your system. You must also specify what
Test personas are meant to help protect existing policies and resources from changes that might accidentally grant erroneous access to data. Learn more
| Label | {{ model.dto.ui.label }} | Sources |
{{ source }}
|
Passport Visas |
{{ claim.type }}: Value: {{ claim.value }} |
|
|---|


Model 1896 is 8 site a gang programmer with standard 1Gbit memory. Based on the architecture of the best seller M1895.
| Programming Target | EPROM, EEPROM, Flash ROM, One chip microcontroller |
|---|---|
| Memory Size | Standard: 1G bit |
| The number of simultaneous programming | Max. number of simultaneous programming 8 pieces (standard 4 socket unit) |
| Operating Mode |
COPY: Copy data from flash ROM to the buffer memory of the programmer. BLANK: Blank check of flash ROM PROGRAM: Program and verify data VERIFY: Verify data in flash ROM and the buffer memory of the programmer CONT: Continuous operation of BLANK+PROGARM+VERIFY ERASE: Erase data in flash ROM SET PROGRAM: Simultaneous programming of different data, divided data programming in 16 bits, divided data programming in 32 bit |
| Data Input | USB, RS232-C, Copy from master ROM |
| Interface | USB, RS232-C |
| Display | 20 x 4 (characters), LCD |
| Power Supply | From 100 to 240V(automatic change), 50 – 66Hz |
| Measurement | 380(W)x100(H)x400(D)mm |
| Weight | 5kg(6.5kg with socket unit) |
| Power Consumption | 30VA(Max.) |
| Recommended Temperature | 5〜35℃ |
業務經理: Joyce Chen
TEL: +886-2-2797-0115 #292
Email: joyce_chen@clpe.com.tw


Module cassandra
Cassandra client library for Lua and ngx_lua (OpenResty).
Compatible with Cassandra 2.0+, Lua 5.1, 5.2, 5.3, LuaJIT, OpenResty.
Support for binary protocols v2 and v3.
Info:
- Release: 0.5.0
- Author: thibaultcha
Session
| Session:execute (query, args, query_options) | Execute a query. |
| Session:batch (queries, query_options) | Execute a batch of queries. |
| Session:set_keyspace (keyspace) | Switch the session's keyspace. |
| Session:set_keep_alive () | Put the underlying sockets into the connection pool and keep them alive. |
| Session:shutdown () | Terminate a session. |
Cassandra
| Cassandra.spawn_session (options) | Spawn a session to connect to the cluster. |
| Cassandra.spawn_cluster (options) | Load the cluster topology. |
| type_serializers | Type serializer shorthands. |
| consistencies | CQL data consistencies. |
| cql_errors | CQL Error types. |
| auth | Authentication providers that can be instanciated and given to a session 's options. |
Session
- Session:execute (query, args, query_options)
-
Execute a query.
The session will choose a coordinator from the cached cluster topology according
to the configured load balancing policy. Once connected to it, it will perform the
given query. This operation is non-blocking in the context of ngx_lua because it
uses the cosocket API (coroutine based).
Queries can have parameters binded to it, they can be prepared, result sets can be paginated, and more, depending on the given
query_options
.If a node is not responding, it will be marked as being down. Since the state of the cluster is shared accross all workers by the ngx.shared.DICT API, all the other active sessions will be aware of it and will not attempt to connect to that node. The configured reconnection policy will decide when it is time to try to connect to that node again.
When selecting a large number of rows, it is a good practise to not increase the
page_size
option, but to iterate over them. It is also useful to build web services. For such usages, this function accepts apaging_state
and anauto_paging
options.Parameters:
- query string The CQL query to execute, possibly with placeholder for binded parameters.
- args table Optional A list of parameters to be binded to the query's placeholders.
- query_options
table
Optional Override the session
s
options.query_options` with the given values, for this request only.
Returns:
-
table
result/rows
: A table describing the result. The content of this table depends on the type of the query. If an error occurred, this value will benil
and a second value describing the error is returned. -
table
error
: A table describing the error that occurred.
Usage:
local res, err = session:execute [[ CREATE KEYSPACE IF NOT EXISTS my_keyspace WITH REPLICATION = {'class': 'SimpleStrategy', 'replication_factor': 1} ]] local rows, err = session:execute("SELECT * FROM system.schema_keyspaces") for i, row in ipairs(rows) do print(row.keyspace_name) end local rows, err = session:execute("SELECT * FROM users WHERE age = ?", {42}, {prepare = true}) for i, row in ipairs(rows) do print(row.username) end -- With auto-pagination for rows, err, page in session:execute("SELECT * FROM users", nil, {page_size = 50, auto_paging = true}) do -- rows contains the retrived rows for this page -- err contains an eventual error and will interrupt the iteration if it is set -- page contains the current page number, starting from 1 end
- Session:batch (queries, query_options)
-
Execute a batch of queries.
The session will chose a coordinator according to the load balancing policy and
execute the batch on it. By default, the session executes a
logged
batch. If given in thequery_options
, the session can also executeunlogged
andcounter
batches. The queries composing a batch can also be prepared.Be weary that depending on the number of partitions involved in the given queries, using batches can overload a coordinator, lead to performance drops and constitute an anti-pattern.
See the documentation on CQL Batches.
Parameters:
- queries table A list of CQL queries that constitutes the batch. Each value can contain either a string, or a table containing the query and its binded parameters.
- query_options
table
Optional Override the session's
options.query_options
with the given values, for this batch only. Can also be used to specify other batch types.
Returns:
-
table
result
: A table describing the result. Batch results are alwaysVOID
results. If an error occurred, this value will benil
and a second value describing the error is returned. -
table
error
: A table describing the error that occurred.
Usage:
-- Basic logged batch local res, err = session:batch({ {"INSERT INTO users(id, name) VALUES(123, 'Alice')"}, {"UPDATE users SET name = 'Alicia' WHERE id = 123"} }) -- With binded parameters and prepared queries local res, err = session:batch({ {"INSERT INTO users(id, name) VALUES(?, ?)", {cassandra.uuid(some_uuid), "Alice"}}, {"UPDATE users SET name = ? WHERE id = ?", {cassandra.uuid(some_uuid), "Alicia"}} }, {prepare = true}) -- Unlogged batch local res, err = session:batch({ {"INSERT INTO users(id, name) VALUES(123, 'Alice')"}, {"UPDATE users SET name = 'Alicia' WHERE id = 123"} }, {logged = false}) -- Counter batch local res, err = session:batch({ {"UPDATE table SET value = value + 1 WHERE key = 'counter'"}, {"UPDATE table SET value = value + 5 WHERE key = 'counter'"}, {"UPDATE table SET value = value + 2 WHERE key = ?", {"counter"}} }, {counter = true})
- Session:set_keyspace (keyspace)
-
Switch the session's keyspace.
All underlying sockets of the session will switch to the given keyspace. Usually, the keyspace is set
by the
keyspace
option given to spawn_session, and it is a better practise to use another session for another keyspace, but sometimes, it might be useful to switch a sesson to another keyspace.Parameters:
- keyspace string The name of the keyspace to switch to.
Returns:
-
boolean
ok
: True if successful, false otherwise. If false, a second value describing the error is returned. -
table
error
: A table describing the error that occurred.
Usage:
local ok, err = session:set_keyspace("my_keyspace") if not ok then -- handle err end
- Session:set_keep_alive ()
-
Put the underlying sockets into the connection pool and keep them alive.
When used inside of ngx_lua, the session will leverage the cosocket API, which maintains
a connection pool of sockets to re-use for better performance. Once the session has finished
its work, calling this function will allow other sessions to re-use the sockets, without having
to open new ones.
See the lua-nginx-module documentation on tcpsocket:setkeepalive().
The underlying tcpsocket:setkeepalive() function will be called with the session
s options values defined in
socketoptions.pooltimeoutand
socketoptions.poolsize. Those values are
nil` by default.When called outside of ngx_lua, the sockets will be closed.
This function does not take any parameters and does not have any return values.
Usage:
-- Execute queries... session:set_keep_alive()
- Session:shutdown ()
-
Terminate a session.
Close all underlying sockets and free all data.
A session closed this way cannot be re-used.
This function does not take any parameters and does not have any return values.
Cassandra
- Cassandra.spawn_session (options)
-
Spawn a session to connect to the cluster.
Sessions are meant to be short-lived and many can be created in parallel. In the context
of ngx_lua, it makes perfect sense for a session to be spawned in a phase handler, and
quickly disposed of by putting the sockets it used back into the cosocket connection pool.
The session will retrieve the cluster topology from the configured shared dict or, if not found, by connecting to one of the optionally given
contact_points
. If you want to pre-load the cluster topology, see spawn_cluster. The created session will use the configured load balancing policy to choose a coordinator from the retrieved cluster topology on each query.Parameters:
- options table The session's options, including the shared dict name and *optionally the contact points
Returns:
-
table
session
: A instanciated session, ready to be used. If an error occurred, this value will benil
and a second value is returned describing the error. -
table
error
: A table describing the error that occurred.
Usage:
access_by_lua_block { local cassandra = require "cassandra" local session, err = cassandra.spawn_session { shm = "cassandra", contact_points = {"<IP_ADDRESS>", "<IP_ADDRESS>"} } if not session then ngx.log(ngx.ERR, tostring(err)) end -- execute query(ies) session:set_keep_alive() }
- Cassandra.spawn_cluster (options)
-
Load the cluster topology.
Iterate over the given
contact_points
and connects to the first one available to load the cluster topology. All peers of the chosen contact point will be retrieved and stored locally so that the load balancing policy can chose one on each request that will be executed.Use this function if you want to retrieve the cluster topology sooner than when you will create your first
Session
.Parameters:
- options table The cluster's options, including the shared dict name and the contact points.
Returns:
-
boolean
ok
: Success of the cluster topology retrieval. If false, a second value will be returned describing the error. -
table
error
: An error in case the operation did not succeed.
Usage:
init_worker_by_lua_block { local cassandra = require "cassandra" local cluster, err = cassandra.spawn_cluster { shm = "cassandra", contact_points = {"<IP_ADDRESS>"} } } access_by_lua_block { local cassandra = require "cassandra" -- The cluster topology is already loaded at this point, -- avoiding latency on your first request. local session, err = cassandra.spawn_session { shm = "cassandra" } }
- type_serializers
-
Type serializer shorthands.
When binding parameters to a query from execute, some
types cannot be infered automatically and will require manual
serialization. Some other times, it can be useful to manually enforce
the type of a parameter.
See the documentation on CQL Data Types.
For this purpose, shorthands for type serialization are available on the
Cassandra
table.Fields:
- unset
Equivalent to the
null
CQL value. Useful to unset a field.cassandra.unset()
- uuid
Serialize a 32 lowercase characters string to a CQL uuid.
cassandra.uuid("123e4567-e89b-12d3-a456-426655440000")
- timestamp
Serialize a 10 digits number into a CQL timestamp.
cassandra.timestamp(1405356926)
- list
cassandra.list({"abc", "def"})
- set
cassandra.set({"abc", "def"})
- map
cassandra.map({foo = "bar"})
- udt CQL UDT.
- tuple CQL tuple.
- inet
CQL inet.
cassandra.inet("<IP_ADDRESS>") cassandra.inet("2001:0db8:85a3:0042:1000:8a2e:0370:7334")
- bigint
CQL bigint.
cassandra.bigint(42000000000)
- double
CQL double.
cassandra.bigint(1.0000000000000004)
- ascii CQL ascii.
- blob CQL blob.
- boolean
CQL boolean.
cassandra.boolean(true)
- counter
CQL counter.
cassandra.counter(1)
- decimal CQL decimal.
- float
CQL float.
cassandra.float(1.618033)
- int
CQL int.
cassandra.float(10)
- text
CQL text.
cassandra.float("hello world")
- timeuuid CQL timeuuid.
- varchar CQL varchar.
- varint CQL varint.
Usage:
session:execute("SELECT * FROM users WHERE id = ?", { cassandra.uuid("123e4567-e89b-12d3-a456-426655440000") }) session:execute("INSERT INTO users(id, emails) VALUES(?, ?)", { 1<EMAIL_ADDRESS><EMAIL_ADDRESS>})
- unset
- consistencies
-
CQL data consistencies.
One of the
options
forSession
and execute is the CQL consistency to use with the queries executed. This table allowes you to specify other consistencies than the defaultONE
, by specifing thequery_options.consistency
option.See the documentation on CQL data consistency.
Fields:
- all
CQL consistency
ALL
.cassandra.consistencies.all
- each_quorum
CQL consistency
EACH_QUORUM
.cassandra.consistencies.each_quorum
- quorum
CQL consistency
QUORUM
.cassandra.consistencies.quorum
- local_quorum
CQL consistency
LOCAL_QUORUM
.cassandra.consistencies.local_quorum
- one
CQL consistency
ONE
.cassandra.consistencies.one
- two
CQL consistency
TWO
.cassandra.consistencies.two
- three
CQL consistency
THREE
.cassandra.consistencies.three
- local_one
CQL consistency
LOCAL_ONE
.cassandra.consistencies.local_one
- any
CQL consistency
ANY
.cassandra.consistencies.any
- serial
CQL consistency
SERIAL
.cassandra.consistencies.seriam
- local_serial
CQL consistency
LOCAL_SERIAL
.cassandra.consistencies.local_serial
Usage:
-- For a session cassandra.spawn_session { query_options = {consistency = cassandra.consistencies.local} } -- For a single query session:execute("SELECT * FROM users", nil, {consistency = cassandra.consistencies.local})
- all
- cql_errors
-
CQL Error types.
The library returns error that are tables. When the error type is
ResponseError
, the table contains a field namedcode
, which describes the CQL error. That code is an hexadecimal value, that can be compared against the constants defined in this table.Fields:
- SERVER something unexpected happened. This indicates a server-side bug.
- PROTOCOL
some client message triggered a protocol violation (for instance a
QUERY
message is sent before aSTARTUP
) - BAD_CREDENTIALS
CREDENTIALS
request failed because Cassandra did not accept the provided credentials. - UNAVAILABLE_EXCEPTION
the query could not be processed with respect to the given concurrency.
(This error could return more informations but it is not yet supported.)
- OVERLOADED the request cannot be processed because the coordinator node is overloaded.
- IS_BOOTSTRAPPING the request was a read request but the coordinator node is bootstrapping.
- TRUNCATE_ERROR error during a truncation error.
- WRITE_TIMEOUT timeout exception during a write request. (This error could return more informations but it is not yet supported.)
- READ_TIMEOUT timeout exception during a read request. (This error could return more informations but it is not yet supported.)
- SYNTAX_ERROR the submitted query has a syntax error.
- UNAUTHORIZED the logged user doesn't have the right to perform the query.
- INVALID the query is syntactically correct but invalid.
- CONFIG_ERROR the query is invalid because of some configuration issue.
- ALREADY_EXISTS the query attempted to create a keyspace or a table that was already existing. (This error could return more informations but it is not yet supported.)
- UNPREPARED Can be thrown while a prepared statement tries to be executed if the provide prepared statement ID is not known by this host. (This error could return more informations but it is not yet supported.)
- auth
-
Authentication providers that can be instanciated and given to a
session
's options.Fields:
- PlainTextProvider
provider for plain text password authentication challenge (like
PasswordAuthenticator
).cassandra.auth.PlainTextProvider("username", "password")
- PlainTextProvider


FlightSurety
Flight
Submit to Oracles
Purchase Flight Insurance
Flight1
Flight2
Flight3
Flight4
Flight5
Flight6
Flight7
Flight8
Flight9
Flight10
Purchase
Claim Repayment
Claim


-- This file is part of the Practicum project. --/>
<-- Copyright 2014 David W. Hogg (NYU). --/>
The Practicum
NYU GSAS PHYS-XXX Fall 2014
Staff
Prof David W. Hogg (NYU Physics; NYU CDS; MPI Astronomie)
- office: 501 Meyer
- office hours: TBD
- contact: [send email]
Aims
This course has several aims, and components corresponding to those
aims:
- Practice teaching physics.
- Read key research literature on teaching and the classroom.
- Experience and reflect on good teaching by others.
- (Discuss responsible conduct of research.)
The last item is unlike the others; it is part of our grant
administration requirements at NYU.
Expectations
Reasonable performance in this class is a pre-requisite to being
assigned teaching positions within the Physics Department. Reasonable
performance in the responsible conduct of research (RCR) part of this
class is a pre-requisite to being supported on any external federal
grant. I will not grade harshly, but I will expect the following of
all students:
- attendance: I will expect every student to come to every
class. If you are going to miss a class for any reason, I expect a
good excuse to be presented to me by email and in
advance of the missed class. The only exceptions are medical
excuses, which can be presented after, but once again I would like
to get these by email.
- written work: I will ask for written reactions to some of
the papers we read, to the classroom visits we organize, and to some
of the in-class presentations. These will not have to be long, but
I expect them to be submitted by the given deadlines, and in
(relatively) good English. If you have trouble with English, find
resources to help you. Good English is important to teaching and
doing research in physics.
Resources
Schedule
<-- standards compliant? -->


What is the Tech Radar?
The Goomer Tech Radar is a list of technologies, complemented by an assessment result, called ring assignment. We use four rings with the following semantics:
- ADOPT — Technologies we have high confidence in to serve our purpose, also in large scale. Technologies with a usage culture in our Goomer production environment, low risk and recommended to be widely used.
- TRIAL — Technologies that we have seen work with success in project work to solve a real problem; first serious usage experience that confirm benefits and can uncover limitations. TRIAL technologies are slightly more risky; some engineers in our organization walked this path and will share knowledge and experiences.
- ASSESS — Technologies that are promising and have clear potential value-add for us; technologies worth to invest some research and prototyping efforts in to see if it has impact. ASSESS technologies have higher risks; they are often brand new and highly unproven in our organisation. You will find some engineers that have knowledge in the technology and promote it, you may even find teams that have started a prototyping effort.
- HOLD — Technologies not recommended to be used for new projects. Technologies that we think are not (yet) worth to (further) invest in. HOLD technologies should not be used for new projects, but usually can be continued for existing projects.
|
What is the purpose?
The Tech Radar is a tool to inspire and support engineering teams at Goomer to pick the best
technologies for new projects; it provides a platform to share knowledge and experience in
technologies, to reflect on technology decisions and continuously evolve our technology landscape.
Based on the pioneering work of ThoughtWorks, our
Tech Radar sets out the changes in technologies that are interesting in software development —
changes that we think our engineering teams should pay attention to and consider using in their
projects. This page is heavily based on
Zalando Tech Radar.
|


D3.js Force-Directed Graph - Nodes' fixed positions
Double Click to save the node and reload the page / Drag & drop and manually reload to save the graph
{{selectedNode.name}}


Loc Nguyen
Loc Nguyen's Academic Network, Vietnam
Email<EMAIL_ADDRESS>Homepage: www.locnguyen.net
| Type | Book |
| Abstract |
Statistics, multivariate data analysis and convex optimization are applied widely in many scientific domains and most analytical techniques are developed based on matrix analysis and matrix calculus because matrix is abstract representation of multivariate data. Although it is slightly confused for us to comprehend their concepts and theories, matrix analysis and calculus give us exciting results which enhance data analysis techniques to be more plentiful and accurate. So the report is survey of matrix analysis and calculus, which includes five main sections such as basic concepts, matrix analysis, matrix derivative, composite derivative, and applications of matrix. Matrix derivative and composite derivative are subjects of matrix calculus. Keywords: Matrix Algebra, Matrix Analysis, Matrix Calculus. |
| In |
Publisher: Lambert Academic Publishing, a trademark of OmniScriptum GmbH & Co. KG. Publication date is December 15, 2015. Address: OmniScriptum GmbH & Co. KG, Heinrich-Böcking-Str. 6-8, Saarbruecken 66121, Germany. Website: https://www.lap-publishing.com Fax: +49 681 93 81 567-9 Email<EMAIL_ADDRESS><EMAIL_ADDRESS>Facebook: https://www.facebook.com/lappublishing Editor: Carolyn Evans |
| Identifiers |
ISBN-13: 978-3-659-69400-4, ISBN-10:<PHONE_NUMBER> EAN:<PHONE_NUMBER>004 Number of pages: 72 |
| Links | https://www.morebooks.de/store/gb/book/matrix-analysis-and-calculus/isbn/978-3-659-69400-4 |
| Citations | Nguyen, L. (2015, December 15). Matrix Analysis and Calculus (1st ed.). (C. Evans, Ed.) Hanoi, Vietnam: Lambert Academic Publishing. Retrieved from https://www.morebooks.de/store/gb/book/matrix-analysis-and-calculus/isbn/978-3-659-69400-4 |
| Cited | |
| Indexed | |
| Metrics | |
| Categories | Mathematics |
| Charged |
Purchase on MoreBooks. Purchase on Amazon. Purchase on Adlibris. |


Creatives and Campaigns
Creatives and Campaigns
SELECT
*
FROM
creatives;
|
Creatives
|
|
id
|
title
|
campaign_id
|
|
1
|
Skrillex Bees!
|
1
|
|
2
|
Cooking With Skrillex
|
1
|
|
3
|
Cooking With LotR Pwniez
|
2
|
SELECT
*
FROM
campaigns;
|
Campaigns
|
|
id
|
name
|
Entry Date
|
|
1
|
Skrillex
|
29/1/13
|
|
2
|
Pwniez
|
1/3/13
|
|
3
|
Cats
|
4/7/12
|
Everything in both tables
SELECT
*
FROM
campaigns, creatives;
| |
| id | title | campaign_id | id_1 | name | Entry Date |
| 1 | Skrillex Bees! | 1 • | • 1 | Skrillex | 29/1/13 |
| 2 | Cooking With Skrillex | 1 • | • 1 | Skrillex | 29/1/13 |
| 3 | Cooking With LotR Pwniez | 2 | • 1 | Skrillex | 29/1/13 |
| 1 | Skrillex Bees! | 1 • | 2 | Pwniez | 1/3/13 |
| 2 | Cooking With Skrillex | 1 • | 2 | Pwniez | 1/3/13 |
| 3 | Cooking With LotR Pwniez | 2 | 2 | Pwniez | 1/3/13 |
| 1 | Skrillex Bees! | 1 • | 3 | Cats | 4/7/12 |
| 2 | Cooking With Skrillex | 1 • | 4 | Cats | 4/7/12 |
| 3 | Cooking With LotR Pwniez | 2 | 4 | Cats | 4/7/12 |
Join on Matching Foreign Key
SELECT
*
FROM
campaigns, creatives
WHERE
creatives.campaign_id = campaigns.id;
| |
| id | title | campaign_id | id_1 | name | Entry Date |
| 1 | Skrillex Bees! | 1 • | • 1 | Skrillex | 29/1/13 |
| 2 | Cooking With Skrillex | 1 • | • 1 | Skrillex | 29/1/13 |
| 3 | Cooking With LotR Pwniez | 2 | • 1 | Skrillex | 29/1/13 |
| 1 | Skrillex Bees! | 1 • | 2 | Pwniez | 1/3/13 |
| 2 | Cooking With Skrillex | 1 • | 2 | Pwniez | 1/3/13 |
| 3 | Cooking With LotR Pwniez | 2 | 2 | Pwniez | 1/3/13 |
| 1 | Skrillex Bees! | 1 • | 3 | Cats | 4/7/12 |
| 2 | Cooking With Skrillex | 1 • | 4 | Cats | 4/7/12 |
| 3 | Cooking With LotR Pwniez | 2 | 4 | Cats | 4/7/12 |
| |
| id | title | campaign_id | id_1 | name | Entry Date |
| 1 | Skrillex Bees! | 1 • | • 1 | Skrillex | 29/1/13 |
| 2 | Cooking With Skrillex | 1 • | • 1 | Skrillex | 29/1/13 |
| 3 | Cooking With LotR Pwniez | 2 | 2 | Pwniez | 1/3/13 |
restrict the columns
SELECT
id,
title,
name,
entry_date
FROM
campaigns,
creatives
WHERE
creatives.campaign_id = campaigns.id;
| |
| id | title | campaign_id | id_1 | name | Entry Date |
| 1 | Skrillex Bees! | 1 • | • 1 | Skrillex | 29/1/13 |
| 2 | Cooking With Skrillex | 1 • | • 1 | Skrillex | 29/1/13 |
| 3 | Cooking With LotR Pwniez | 2 | 2 | Pwniez | 1/3/13 |
| |
| id | title | name | Entry Date |
| 1 | Skrillex Bees! | Skrillex | 29/1/13 |
| 2 | Cooking With Skrillex | Skrillex | 29/1/13 |
| 3 | Cooking With LotR Pwniez | Pwniez | 1/3/13 |
only Titles that Start With Cooking
SELECT
id,
title,
name,
entry_date
FROM
campaigns,
creatives
WHERE
creatives.campaign_id = campaigns.id
AND
title ILIKE 'Cooking%;
| |
| id | title | name | Entry Date |
| 1 | Skrillex Bees! | Skrillex | 29/1/13 |
| 2 | Cooking With Skrillex | Skrillex | 29/1/13 |
| 3 | Cooking With LotR Pwniez | Pwniez | 1/3/13 |
| |
| id | title | name | Entry Date |
| 2 | Cooking With Skrillex | Skrillex | 29/1/13 |
| 3 | Cooking With LotR Pwniez | Pwniez | 1/3/13 |
only creatives Added in Last 24 Hours
SELECT
id,
title,
name,
entry_date
FROM
campaigns,
creatives
WHERE
creatives.campaign_id = campaigns.id
AND
entry_date >= now() - interval '24 hours';
| |
| id | title | name | Entry Date |
| 1 | Skrillex Bees! | Skrillex | 29/1/13 |
| 2 | Cooking With Skrillex | Skrillex | 29/1/13 |
| 3 | Cooking With LotR Pwniez | Pwniez | 1/3/13 |
| |
| id | title | name | Entry Date |
| 3 | Cooking With LotR Pwniez | Pwniez | 1/3/13 |
Now With Added OOP-Goodness
Now With Added OOP-Goodness
SELECT * FROM creatives;
| Creatives |
| id | title | discriminator | campaign_id |
| 1 | Skrillex Bees! | ViralCreative | 1 • |
| 2 | Cooking With Skrillex | EngagementCreative | 1 • |
| 3 | Cooking With LotR Pwniez | ViralCreative | 2 • |
SELECT * FROM campaigns;
| Campaigns |
| id | name | Entry Date |
| • 1 | Skrillex | 29/1/13 |
| • 2 | Pwniez | 1/3/13 |
| 3 | Cats | 4/7/12 |
only Viral Creatives
SELECT
id,
title,
name,
discriminator,
entry_date
FROM
campaigns,
creatives
WHERE
creatives.campaign_id = campaigns.id
AND
discriminator = 'ViralCreative';
| |
| id | title | discriminator | name | Entry Date |
| 1 | Skrillex Bees! | ViralCreative | Skrillex | 29/1/13 |
| 2 | Cooking With Skrillex | EngagementCreative | Skrillex | 29/1/13 |
| 3 | Cooking With LotR Pwniez | ViralCreative | Pwniez | 1/3/13 |
| |
| id | title | discriminator | name | Entry Date |
| 1 | Skrillex Bees! | ViralCreative | Skrillex | 29/1/13 |
| 3 | Cooking With LotR Pwniez | ViralCreative | Pwniez | 1/3/13 |


Table of Contents
Machine Learning Basics
- Introduction and Definitions:
- Two Approaches to Statistics:
- Frequentest Estimators
- Bayesian Inference
- The Design Matrix:
A common way for describing a dataset where it is a matrix containing a different example in each row. Each column of the matrix corresponds to a different feature.
- Two Approaches to Statistics:
- Learning Algorithms:
- Learning:
A computer program is said to learn from experience \(E\) with respect to some class of tasks \(T\) and performance measure \(P\), if its performance at tasks in \(T\), as measured by \(P\), improves with experience \(E\). - The Task \(T\):
- Classification:
A task where the computer program is asked to specify which of \(k\) categories some input belongs to.
To solve this task, the learning algorithm is usually asked to produce a function \(f:\mathbb{R}^n \rightarrow {1, . . . , k}\).
When \(y=f(x)\), the model assigns an input described by vector \(x\) to a category identified by numeric code \(y\).e.g. Object Recognition
-
Classification with Missing Inputs:
Classification becomes more challenging if the computer program is not guaranteed that every measurement in its input vector will always be provided.
To solve this task, rather than providing a single classification function (as in the normal classification case), the learning algorithm must learn a set of functions, each corresponding to classifying \(x\) with a different subset of its inputs missing.One way to efficiently define such a large set of functions is to learn a probability distribution over all the relevant variables, then solve the classification task by marginalizing out the missing variables.
With \(n\) input variables, we can now obtain all \(2^n\) different classification functions needed for each possible set of missing inputs, but the computer program needs to learn only a single function describing the joint probability distribution.e.g. Medical Diagnosis (where some tests weren’t conducted for any reason)
- Regression:
A computer is asked to predict a numerical value given some input.
To solve this task, the learning algorithm is asked to output a function \(f:\mathbb{R}^n \rightarrow R\)e.g. Object Localization
- Transcription:
In this type of task, the machine learning system is asked to observe a relatively unstructured representation of some kind of data and transcribe the information into discrete textual form.e.g. OCR
- Machine Translation:
In a machine translation task, the input already consists of a sequence of symbols in some language, and the computer program must convert this into a sequence of symbols in another language.e.g. Google Translate
- Structured Output:
Structured output tasks involve any task where the output is a vector (or other data structure containing multiple values) with important relationships between the different elements.
This is a broad category and subsumes the transcription and translation tasks described above, as well as many other tasks.
These tasks are called structured output tasks because the program must output several values that are all tightly interrelated. For example, the words produced by an image captioning program must form a valid sentence.e.g. Syntax Parsing, Image Segmentation
- Anomaly Detection:
In this type of task, the computer program sifts through a set of events or objects and ﬂags some of them as being unusual or atypical.e.g. Insider Trading Detection
- Synthesis and Sampling:
In this type of task, the machine learning algorithm is asked to generate new examples that are similar to those in the training data.
This is a kind of structured output task, but with the added qualification that there is no single correct output for each input, and we explicitly desire a large amount of variation in the output, in order for the output to seem more natural and realistic.e.g. Image Synthesis, Speech Synthesis
- Imputation:
In this type of task, the machine learning algorithm is given a new example \(x \in \mathbb{R}^n\), but with some entries \(x_i\) of \(x\) missing. The algorithm must provide a prediction of the values of the missing entries. - Denoising:
In this type of task, the machine learning algorithm is given as input a corrupted example \(\tilde{x} \in \mathbb{R}^n\) obtained by an unknown corruption process from a clean example \(x \in \mathbb{R}^n\). The learner must predict the clean example \(x\) from its corrupted version \(\tilde{x}\), or more generally predict the conditional probability distribution \(p(x |\tilde{x})\).e.g. Signal Reconstruction, Image Artifact Removal
- Density (Probability Mass Function) Estimation:
In the density estimation problem, the machine learning algorithm is asked to learn a function \(p_\text{model}: \mathbb{R}^n \rightarrow R\), where \(p_\text{model}(x)\) can be interpreted as a probability density function (if \(x\) is continuous) or a probability mass function (if \(x\) is discrete) on the space that the examples were drawn from.
To do such a task well, the algorithm needs to learn the structure of the data it has seen. It must know where examples cluster tightly and where they are unlikely to occur.
Most of the tasks described above require the learning algorithm to at least implicitly capture the structure of the probability distribution (i.e. it can be computed but we don’t have an equation for it). Density estimation enables us to explicitly capture that distribution.
In principle,we can then perform computations on that distribution to solve the other tasks as well.
For example, if we have performed density estimation to obtain a probability distribution p(x), we can use that distribution to solve the missing value imputation task. Equivalently, if a value \(x_i\) is missing, and all the other values, denoted \(x_{−i}\), are given, then we know the distribution over it is given by \(p(x_i| x_{−i})\).
In practice, density estimation does not always enable us to solve all these related tasks, because in many cases the required operations on p(x) are computationally intractable.e.g. Language Modeling
- A Lot More:
Their are many more tasks that could be defined for and solved by Machine Learning. However, this is a list of the most common problems, which have a well-known set of methods for handling them.
- Classification:
- The Performance Measure \(P\):
A quantitative measure of the performance of a machine learning algorithm.
We often use accuracy or error rate.
- Learning:
- Machine Learning Methods:
-
- Instance-based algorithm
- K-nearest neighbors algorithm (KNN)
- Learning vector quantization (LVQ)
- Self-organizing map (SOM)
- Regression analysis
- Logistic regression
- Ordinary least squares regression (OLSR)
- Linear regression
- Stepwise regression
- Multivariate adaptive regression splines (MARS)
- Regularization algorithm
- Classifiers
- Dimensionality reduction
- Canonical correlation analysis (CCA)
- Factor analysis
- Feature extraction
- Feature selection
- Independent component analysis (ICA)
- Linear discriminant analysis (LDA)
- Multidimensional scaling (MDS)
- Non-negative matrix factorization (NMF)
- Partial least squares regression (PLSR)
- Principal component analysis (PCA)
- Principal component regression (PCR)
- Projection pursuit
- Sammon mapping
- t-distributed stochastic neighbor embedding (t-SNE)
- Ensemble learning
- AdaBoost
- Boosting
- Bootstrap aggregating (Bagging)
- Ensemble averaging – process of creating multiple models and combining them to produce a desired output, as opposed to creating just one model. Frequently an ensemble of models performs better than any individual model, because the various errors of the models “average out.”
- Gradient boosted decision tree (GBDT)
- Gradient boosting machine (GBM)
- Random Forest
- Stacked Generalization (blending)
- Meta learning
- Reinforcement learning
- Supervised learning
- AODE
- Artificial neural network
- Association rule learning algorithms
- Case-based reasoning
- Gaussian process regression
- Gene expression programming
- Group method of data handling (GMDH)
- Inductive logic programming
- Instance-based learning
- Lazy learning
- Learning Automata
- Learning Vector Quantization
- Logistic Model Tree
- Minimum message length (decision trees, decision graphs, etc.)
- Probably approximately correct learning (PAC) learning
- Ripple down rules, a knowledge acquisition methodology
- Symbolic machine learning algorithms
- Support vector machines
- Random Forests
- Ensembles of classifiers
- Ordinal classification
- Information fuzzy networks (IFN)
- Conditional Random Field
- ANOVA
- Quadratic classifiers
- k-nearest neighbor
- Boosting
- SPRINT
- Bayesian networks
- Hidden Markov models
- Bayesian statistics
- Bayesian knowledge base
- Naive Bayes
- Gaussian Naive Bayes
- Multinomial Naive Bayes
- Averaged One-Dependence Estimators (AODE)
- Bayesian Belief Network (BBN)
- Bayesian Network (BN)
- Decision tree algorithms
- Decision tree
- Classification and regression tree (CART)
- Iterative Dichotomiser 3 (ID3)
- C4.5 algorithm
- C5.0 algorithm
- Chi-squared Automatic Interaction Detection (CHAID)
- Decision stump
- Conditional decision tree
- ID3 algorithm
- Random forest
- SLIQ
- Linear classifier
- Unsupervised learning
- Artificial neural network
- Association rule learning
- Hierarchical clustering
- Cluster analysis
- Anomaly detection
- Semi-supervised learning
- Active learning – special case of semi-supervised learning in which a learning algorithm is able to interactively query the user (or some other information source) to obtain the desired outputs at new data points.[5][6]
- Generative models
- Low-density separation
- Graph-based methods
- Co-training
- Transduction
- Deep learning
- Other machine learning methods and problems
- Anomaly detection
- Association rules
- Bias-variance dilemma
- Classification
- Clustering
- Data Pre-processing
- Empirical risk minimization
- Feature engineering
- Feature learning
- Learning to rank
- Occam learning
- Online machine learning
- PAC learning
- Regression
- Reinforcement Learning
- Semi-supervised learning
- Statistical learning
- Structured prediction
- Unsupervised learning
- VC theory
- Instance-based algorithm
-
-
Learning vs Optimization:
Generalization: is the ability to perform well on previously unobserved inputs.
Generalization (Test) Error: is defined as the expected value of the error on a new input.Learning vs Optimization:
- The problem of Reducing the training error on the training set is one of optimization.
- The problem of Reducing the training error, as well as, the generalization (test) error is one of learning.
-
Statistical Learning Theory:
It is a framework that, under certain assumptions, allows us to study the question of “ How can we affect performance on the test set when we can observe only the training set?”Assumptions:
- The training and test data are generated by a probability distribution over datasets called the data-generating process.
- The i.i.d. assumptions:
- The examples in each dataset are independent from each other
- The training set and test set are identically distributed (drawn from the same probability distribution as each other)
This assumption enables us to describe the data-generating process with a probability distribution over a single example. The same distribution is then used to generate every train example and every test example.
- We call that shared underlying distribution the data-generating distribution, denoted \(p_{\text{data}}\)
This probabilistic framework and the i.i.d. assumptions enable us to mathematically study the relationship between training error and test error.
-
Capacity, Overfitting, and Underfitting:
The ML process:
We sample the training set, then use it to choose the parameters to reduce training set error, then sample the test set.
Under this process, the expected test error is greater than or equal to the expected value of training error.The factors determining how well a machine learning algorithm will perform are its ability to:
- Make the training error small
- Make the gap between training and test error small
These two factors correspond to the two central challenges in machine learning: underfitting and overfitting.
Underfitting: occurs when the model is not able to obtain a sufficiently low error value on the training set.
Overfitting: occurs when the gap between the training error and test error is too large.We can control whether a model is more likely to overfit or underfit by altering its capacity.
Capacity: a models capacity is its ability to fit a wide variety of functions.- Models with low capacity may struggle to fit the training set.
- Models with high capacity can overfit by memorizing properties of the training set that do not serve them well on the test set.
One way to control the capacity of a learning algorithm is by choosing its hypothesis space.
Hypothesis Space: the set of functions that the learning algorithm is allowed to select as being the solution.Statistical learning theory provides various means of quantifying model capacity.Among these, the most well known is the Vapnik-Chervonenkis (VC) dimension.
The VC Dimension: is defined as being the largest possible value of \(m\) for which there exists a training set of \(m\) different \(\mathbf{x}\) points that the classifier can label arbitrarily.
It measure the capacity of a binary classifier.Quantifying the capacity of the model enables statistical learning theory to make quantitative predictions. The most important results in statistical learning theory show that the discrepancy between training error and generalization error is bounded from above by a quantity that grows as the model capacity grows but shrinks as the number of training examples increases.
Effective capacity and Representational Capacity…
- Regularization:
Regularization is a (more general) way of controlling a models capacity by allowing us to express preference for one function over another in the same hypothesis space; instead of including or excluding members from the hypothesis space completely.We can think of excluding a function from a hypothesis space as expressing an infinitely strong preference against that function.
Regularization can be defined as any modification we make to a learning algorithm that is intended to reduce its generalization error but not its training error.
Example: Weight Decay
It is a regularization form that adds the \(L^2\) norm of the weights to the cost function; allowing us to express preference for smaller weights. It is controlled by a hyperparameter \(\lambda\).$$J(\boldsymbol{w})=\mathrm{MSE}_ {\mathrm{train}}+\lambda \boldsymbol{w}^{\top} \boldsymbol{w}$$
This gives us solutions that have a smaller slope, or that put weight on fewer of the features.
More generally, the regularizer penalty of weight decay is:
$$\Omega(\boldsymbol{w})=\boldsymbol{w}^{\top} \boldsymbol{w}$$
-
Point Estimation:
Point estimation is the attempt to provide the single “best” prediction of some quantity of interest. In general the quantity of interest can be a single parameter or a vector of parameters in some parametric model, such as the weights in linear regression, but it can also be a whole function.
- Estimators:
A Point Estimator or statistic is any function of the data:$$\hat{\boldsymbol{\theta}}_{m}=g\left(\boldsymbol{x}^{(1)}, \ldots, \boldsymbol{x}^{(m)}\right)$$
such that a good estimator is a function whose output is close to the true underlying \(\theta\) that generated the training data.
We assume that the true \(\boldsymbol{\theta}\) is fixed, and that \(\hat{\boldsymbol{\theta}}\) is a function of the data, which is drawn from a random process, making \(\hat{\boldsymbol{\theta}}\) a random variable.
Thus, a point estimator is a random variable.Function Estimation/Approximation refers to estimation of the relationship between input and target data.
I.E. We are trying to predict a variable \(y\) given an input vector \(x\), and we assume that there is a function \(f(x)\) that describes the approximate relationship between \(y\) and \(x\).
If we assume that: \(y = f(x) + \epsilon\), where \(\epsilon\) is the part of \(y\) that is not predictable from \(x\); then we are interested in approximating \(f\) with a model or estimate \(\hat{f}\).Function estimation is really just the same as estimating a parameter \(\boldsymbol{\theta}\); the function estimator \(\hat{f}\) is simply a point estimator in function space.
Linear Regression and Polynomial Regression both illustrate scenarios that may be interpreted as either estimating a parameter \(\boldsymbol{w}\) or estimating a function \(f\) mapping from \(\boldsymbol{x}\) to \(y\).
- Estimators as statistics and their probability distributions
- Estimators in ML
- Plain and Simple Estimators (TF)
Notes:
- weights \(\boldsymbol{w}\) == parameters \(\theta\)
- parameter/point estimation == function estimation
Linear Regression and Polynomial Regression both illustrate scenarios that may be interpreted as either estimating a parameter \(\boldsymbol{w}\) or estimating a function \(f\) mapping from \(\boldsymbol{x}\) to \(y\). - we can use the theory and tools for analyzing point estimators and apply them to function estimators (e.g. bias/var decomp etc.)
- Properties of Estimators - Bias and Variance:
The Bias of an estimator is:$$ \operatorname{bias}\left(\hat{\boldsymbol{\theta}}_{m}\right)=\mathbb{E}\left(\hat{\boldsymbol{\theta}}_{m}\right)-\boldsymbol{\theta} $$
where the expectation is over the data (seen as samples from a random variable) and \(\theta\) is the true underlying value of \(\theta\) used to define the data-generating distribution.
- Unbiased Estimators: An estimator \(\hat{\boldsymbol{\theta}}_{m}\) is said to be unbiased if \(\operatorname{bias}\left(\hat{\boldsymbol{\theta}}_{m}\right)=\mathbf{0}\), which implies that \(\mathbb{E}\left(\hat{\boldsymbol{\theta}}_{m}\right)=\boldsymbol{\theta}\).
- Asymptotically Unbiased Estimators: An estimator is said to be asymptotically unbiased if \(\lim _{m \rightarrow \infty} \operatorname{bias}\left(\hat{\boldsymbol{\theta}}_{m}\right)=\mathbf{0},\) which implies that \(\lim _{m \rightarrow \infty} \mathbb{E}\left(\hat{\boldsymbol{\theta}}_{m}\right)=\boldsymbol{\theta}\)
The Variance of an estimator is a way to measure how much we expect the estimator to vary as a function of the data sample, defined, simply, as the variance over the training set random variable \(\hat{\theta}\):
$$ \operatorname{Var}(\hat{\theta}) $$
The Standard Error \(\operatorname{SE}(\hat{\theta})\), of an estimator, is the square root of the variance.
-
E.g. The Standard Error of the (Sample) Mean:
\(\operatorname{SE}\left(\hat{\mu}_{m}\right)=\sqrt{\operatorname{Var}\left[\frac{1}{m} \sum_{i=1}^{m} x^{(i)}\right]}=\frac{\sigma}{\sqrt{m}}\)
Where \(\sigma^2\) is the true variance of the samples \(x^i\).$$\begin{aligned} \operatorname{Var}\left[\overline{X}_{n}\right] &=\operatorname{Var}\left[\frac{1}{n} \sum_{i=1}^{n} X_{i}\right] \\ &=\frac{1}{n^{2}} \operatorname{Var}\left[\sum_{i=1}^{n} X_{i}\right] \\ &=\frac{1}{n^{2}} \sum_{i=1}^{n} \operatorname{Var}\left[X_{i}\right] \\ &=\frac{1}{n^{2}} \sum_{i=1}^{n} \sigma^{2} \\ &=\frac{1}{n^{2}} n \sigma^{2}=\frac{\sigma^{2}}{n} \end{aligned}$$
Unfortunately, neither the square root of the sample variance nor the square root of the unbiased estimator of the variance provide an unbiased estimate of the standard deviation.
- Generalization Error from Standard Error (of the mean):
We often estimate the generalization error by computing the sample mean of the error on the test set.
Taking advantage of the central limit theorem, which tells us that the mean will be approximately distributed with a normal distribution, we can use the standard error to compute the probability that the true expectation falls in any chosen interval.
For example, the \(95\%\) confidence interval centered on the mean \(\hat{\mu}_ {m}\) is:$$\left(\hat{\mu}_{m}-1.96 \mathrm{SE}\left(\hat{\mu}_{m}\right), \hat{\mu}_{m}+1.96 \mathrm{SE}\left(\hat{\mu}_{m}\right)\right)$$
under the normal distribution with mean \(\hat{\mu}_{m}\) and variance \(\mathrm{SE}\left(\hat{\mu}_{m}\right)^{2}\).
We say that algorithm \(\boldsymbol{A}\) is better than algorithm \(\boldsymbol{B}\) if the <upper bound of the \(95\%\) confidence interval for the error of algorithm \(\boldsymbol{A}\)> is less than the <lower bound of the \(95\%\) confidence interval for the error of algorithm \(\boldsymbol{B}\)>.
- The Bias Variance Trade-off:
Bias and variance measure two different sources of error in an estimator:- Bias: measures the expected deviation from the true value of the function or parameter
- Variance: provides a measure of the deviation from the expected estimator value that any particular sampling of the data is likely to cause
Evaluating Models - Trading off Bias and Variance:
- The most common way to negotiate this trade-off is to use cross-validation
- Alternatively, we can also compare the mean squared error (MSE) of the estimates:
$$\begin{aligned} \mathrm{MSE} &=\mathbb{E}\left[\left(\hat{\theta}_{m}-\theta\right)^{2}\right] \\ &=\operatorname{Bias}\left(\hat{\theta}_{m}\right)^{2}+\operatorname{Var}\left(\hat{\theta}_{m}\right) \end{aligned}$$
The MSE measures the overall expected deviation — in a squared error sense — between the estimator and the true value of the parameter \(\theta\).
- Properties of Estimators - Consistency:
Consistency is a property implying that as the number of data points \(m\) in our dataset increases, our point estimates converge to the true value of the corresponding parameters. Formally:$$\mathrm{plim}_{m \rightarrow \infty} \hat{\theta}_{m}=\theta$$
Where:
\({\text { The symbol plim indicates convergence in probability, meaning that for any } \epsilon>0,}\)$${P\left(\vert\hat{\theta}_ {m}-\theta \vert>\epsilon\right) \rightarrow 0 \text { as } m \rightarrow \infty}$$
Sometimes referred to as Weak Consistency
Strong Consistency applies to almost sure convergence of \(\hat{\theta}\) to \(\theta\).
Consistency and Asymptotic Bias:
- Consistency ensures that the bias induced by the estimator diminishes as the number of data examples grows.
- However, asymptotic unbiasedness does not imply consistency
- It is also useful to note that by Chebyshev’s inequality, if the variance tends to zero then asymptotic unbiasedness implies consistency stex.
Consistency implies both unbiasedness and low variance and therefore, unbiasedness alone is not sufficient to imply consistency. -
Maximum Likelihood Estimation (MLE):
MLE is a method/principle from which we can derive specific functions that are good estimators for different models.Likelihood is the probability of the data given the parameters of the model
Let \(\mathbb{X}=\left\{\boldsymbol{x}^{(1)}, \ldots, \boldsymbol{x}^{(m)}\right\}\) be a set of \(m\) examples drawn independently from the true but unknown data-generating distribution \(p_{\text{data}}(\mathbf{x})\), and let \(p_{\text{model}}(\mathbf{x} ; \boldsymbol{\theta})\) be a parametric family of probability distributions over the same space indexed by \(\boldsymbol{\theta}\)1,
The Maximum Likelihood Estimator for \(\boldsymbol{\theta}\) is:$$\begin{aligned} \boldsymbol{\theta}_{\mathrm{ML}} &=\underset{\boldsymbol{\theta}}{\arg \max } \: p_{\text{model}}(\mathbb{X} ; \boldsymbol{\theta}) \\ &=\underset{\boldsymbol{\theta}}{\arg \max } \prod_{i=1}^{m} p_{\text{model}}\left(\boldsymbol{x}^{(i)} ; \boldsymbol{\theta}\right) \end{aligned}$$
We take the \(log\) for numerical stability:
$$\boldsymbol{\theta}_{\mathrm{ML}}=\underset{\boldsymbol{\theta}}{\arg \max } \sum_{i=1}^{m} \log p_{\text{model}}\left(\boldsymbol{x}^{(i)} ; \boldsymbol{\theta}\right) \tag{5.58}$$
Because the \(\text { arg max }\) does not change when we rescale the cost function, we can divide by \(m\) to obtain a version of the criterion that is expressed as an expectation with respect to the empirical distribution \(\hat{p}_ {\text{data}}\) defined by the training data:
$$\boldsymbol{\theta}_{\mathrm{ML}}=\underset{\boldsymbol{\theta}}{\arg \max } \: \underset{\mathbf{x} \sim \hat{p}_\text{data}}{\mathbb{E}} [\log p_{\text{model}}(\boldsymbol{x} ; \boldsymbol{\theta})] \tag{5.59}$$
MLE as Minimizing KL-Divergence between the Empirical dist. and the model dist.:
We can interpret maximum likelihood estimation as minimizing the dissimilarity between the empirical distribution \(\hat{p}_ {\text{data}}\), defined by the training set, and the model distribution, with the degree of dissimilarity between the two measured by the KL divergence.- The KL-divergence is given by:
$$D_{\mathrm{KL}}\left(\hat{p}_{\text{data}} \| p_{\text{model}}\right)=\mathbb{E}_{\mathbf{x} \sim \hat{p}_{\text{data}}}\left[\log \hat{p}_{\text{data}}(\boldsymbol{x})-\log p_{\text{model}}(\boldsymbol{x})\right] \tag{5.60}$$
The term on the left is a function only of the data-generating process, not the model. This means when we train the model to minimize the KL divergence, we need only minimize:
$$-\mathbb{E}_{\mathbf{x} \sim \hat{p}_{\text{data}}}\left[\log p_{\text{model}}(\boldsymbol{x})\right] \tag{5.61}$$
which is of course the same as the maximization in equation \(5.59\).
Minimizing this KL-divergence corresponds exactly to minimizing the cross-entropy between the distributions.
Any loss consisting of a negative log-likelihood is a cross-entropy between the empirical distribution defined by the training set and theprobability distribution defined by model.
E.g. MSE is the cross-entropy between the empirical distribution and a Gaussian model.We can thus see maximum likelihood as an attempt to make the model distribution match the empirical distribution \(\hat{p} _ {\text{data}}\)2.
Maximum likelihood thus becomes minimization of the negative log-likelihood(NLL), or equivalently, minimization of the cross-entropy3.
Conditional Log-Likelihood (MLE for Supervised Learning):
The maximum likelihood estimator can readily be generalized to estimate a conditional probability \(P(\mathbf{y} | \mathbf{x} ; \boldsymbol{\theta})\) in order to predict \(\mathbf{y}\) given \(\mathbf{x}\). If \(X\) represents all our inputs and \(Y\) all our observed targets, then the conditional maximum likelihood estimator is:$$\boldsymbol{\theta}_ {\mathrm{ML}}=\underset{\boldsymbol{\theta}}{\arg \max } \: P(\boldsymbol{Y} | \boldsymbol{X} ; \boldsymbol{\theta}) \tag{5.62}$$
and the log-likelihood estimator is:
$$\boldsymbol{\theta}_{\mathrm{ML}}=\underset{\boldsymbol{\theta}}{\arg \max } \sum_{i=1}^{m} \log P\left(\boldsymbol{y}^{(i)} | \boldsymbol{x}^{(i)} ; \boldsymbol{\theta}\right) \tag{5.63}$$
Properties of Maximum Likelihood Estimator:
The main appeal of the maximum likelihood estimator is that it can be shown to be the best estimator asymptotically, as the number of examples \(m \rightarrow \infty\), in terms of its rate of convergence as \(m\) increases.- Consistency: as the number of training examples approaches infinity, the maximum likelihood estimate of a parameter converges to the true value of the parameter, under the following conditions:
- The true distribution \(p_{\text{data}}\) must lie within the model family \(p_{\text{model}}(\cdot ; \boldsymbol{\theta})\). Otherwise, no estimator can recover \(p_{\text{data}}\).
- The true distribution \(p_{\text{data}}\) must correspond to exactly one value of \(\boldsymbol{\theta}\). Otherwise, maximum likelihood can recover the correct \(p_{\text{data}}\) but will not be able to determine which value of \(\boldsymbol{\theta}\) was used by the data-generating process.
- Statistical Efficiency: meaning that one consistent estimator may obtain lower generalization error for a fixed number of samples \(m\), or equivalently, may require fewer examples to obtain a fixed level of generalization error.4
The Cramér-Rao lower bound shows that no consistent estimator has a lower MSE than the maximum likelihood estimator.
- The KL-divergence is given by:
- Maximum A Posteriori (MAP) Estimation:
The MAP estimate chooses the point of maximal posterior probability by allowing the prior to influence the choice of the point estimate:$$\boldsymbol{\theta}_ {\mathrm{MAP}}=\underset{\boldsymbol{\theta}}{\arg \max } p(\boldsymbol{\theta} | \boldsymbol{x})=\underset{\boldsymbol{\theta}}{\arg \max } \log p(\boldsymbol{x} | \boldsymbol{\theta})+\log p(\boldsymbol{\theta}) \tag{5.79}$$
Many regularized estimation strategies, such as maximum likelihood learning regularized with weight decay, can be interpreted as making the MAP approximation to Bayesian inference.
E.g. MAP Bayesian inference with a Gaussian prior on the weights corresponds to weight decay Regularization:
consider a linear regression model with a Gaussian prior on the weights \(\mathbf{w}\). If this prior is given by \(\mathcal{N}\left(\boldsymbol{w} ; \mathbf{0}, \frac{1}{\lambda} \boldsymbol{I}^{2}\right)\), then the log-prior term in equation \(5.79\) is proportional to the familiar \(\lambda w^{T} w\) weight decay penalty, plus a constant.This view applies when the regularization consists of adding an extra term to the objective function that corresponds to \(\log p(\boldsymbol{\theta})\) (i.e. logarithm of a probability distribution).
The Mathematics of Neural Networks
- Derivative:
The derivative of a function is the amount that the value of a function changes when the input changes by an \(\epsilon\) amount:$$f'(a)=\lim_{h\to 0}{\frac {f(a+h)-f(a)}{h}}. \\ \text{i.e. } f(x + \epsilon)\approx f(x)+\epsilon f'(x) $$
The Chain Rule is a way to compute the derivative of composite functions.
If \(y = f(x)\) and \(z = g(y)\):$$\dfrac{\partial z}{\partial x} = \dfrac{\partial z}{\partial y} \dfrac{\partial y}{\partial x}$$
-
Gradient: (Vector in, Scalar out)
Gradients generalize derivatives to scalar functions of several variables
Property: the gradient of a function \(\nabla f(x)\) points in the direction of steepest ascent from \(x\).
-
- The Jacobian: (Vector in, Vector out)
- The Jacobian of \(f: \mathbb{R}^n \rightarrow \mathbb{R}^m\) is a matrix of first-order partial derivatives of a vector-valued function:
- The Chain Rule:
Let \(f : \mathbb{R}^N \rightarrow \mathbb{R}^M\) and \(g : \mathbb{R}^M \rightarrow \mathbb{R}^ K\); and let \(x \in \mathbb{R}^N, y \in \mathbb{R}^M\), and \(z \in \mathbb{R}^K\) with \(y = f(x)\) and \(z = g(y)\): - \[\dfrac{\partial z}{\partial x} = \dfrac{\partial z}{\partial y} \dfrac{\partial y}{\partial x}\]
- where, \(\dfrac{\partial z}{\partial y} \in \mathbb{R}^{K \times M}\) matrix, \(\dfrac{\partial y}{\partial x} \in \mathbb{R}^{M \times N}\) matrix, and \(\dfrac{\partial z}{\partial x} \in \mathbb{R}^{K \times N}\) matrix;
the multiplication of \(\dfrac{\partial z}{\partial y}\) and \(\dfrac{\partial y}{\partial x}\) is a matrix multiplication.
-
- The Generalized Jacobian: (Tensor in, Tensor out)
- A Tensor is a D-dimensional grid of number.
- Suppose that \(f: \mathbb{R}^{N_1 \times \cdots \times N_{D_x}} \rightarrow \mathbb{R}^{M_1 \times \cdots \times M_{D_y}}\).
If \(y = f(x)\) then the derivative \(\dfrac{\partial y}{\partial x}\) is a generalized Jacobian - an object with shape: - \[(M_1 \times \cdots \times M_{D_y}) \times (N_1 \times \cdots \times N_{D_x})\]
-
we can think of the generalized Jacobian as generalization of a matrix, where each “row” has the same shape as \(y\) and each “column” has the same shape as \(x\).
- Just like the standard Jacobian, the generalized Jacobian tells us the relative rates of change between all elements of \(x\) and all elements of \(y\):
- \[(\dfrac{\partial y}{\partial x})_{i,j} = \dfrac{\partial y_i}{\partial x_j} \in \mathbb{R}\]
- Just as the derivative, the generalized Jacobian gives us the relative change in \(y\) given a small change in \(x\):
- \[f(x + \delta x)\approx f(x)+ f'(x) \delta x = y + \dfrac{\partial y}{\partial x}\delta x\]
- where now, \(\delta x\) is a tensor in \(\mathbb{R}{N_1 \cdots N_{d_x}}\) and \(\dfrac{\partial y}{\partial x}\) is a generalized matrix in \(\mathbb{R}^{(M_1 \times \cdots \times M_{D_y}) \times (N_1 \times \cdots \times N_{D_x})}\).
The product \(\dfrac{\partial y_i}{\partial x_j} \delta x\) is, therefore, a generalized matrix-vector multiply, which results in a tensor in \(\mathbb{R}^{M_1 \times \cdots \times M_{D_y}}\). - The generalized matrix-vector multiply follows the same algebraic rules as a traditional matrix-vector multiply:
- The Chain Rule:
-
- The Hessian:
- The Hessian Matrix of a scalar function \(f: \mathbb{R}^d \rightarrow \mathbb{R}\) is a matric of second-order partial derivatives:
- Properties:
- The Hessian matrix is symmetric - since we usually work with smooth/differentiable functions - due to Clairauts Theorem.
Clairauts Theorem: if the partial derivatives are continuous, the order of differentiation can be interchanged
- The Hessian is used in some optimization algorithms such as Newton’s method
- It is expensive to calculate but can drastically reduce the number of iterations needed to converge to a local minimum by providing information about the curvature of \(f\)
- The Hessian matrix is symmetric - since we usually work with smooth/differentiable functions - due to Clairauts Theorem.
-
- Matrix Calculus:
- Important Identities:
- \[{\frac {\partial {\mathbf {a}}^{\top }{\mathbf {x}}}{\partial {\mathbf {x}}}}={\frac {\partial {\mathbf {x}}^{\top }{\mathbf {a}}}{\partial {\mathbf {x}}}}= a \\ {\frac {\partial {\mathbf {x}}^{\top }{\mathbf {A}}{\mathbf {x}}}{\partial {\mathbf {x}}}}= ({\mathbf {A}}+{\mathbf {A}}^{\top }){\mathbf {x}} \\ {\frac {\partial {\mathbf {x}}^{\top }{\mathbf {A}}{\mathbf {x}}}{\partial {\mathbf {x}}}}= 2{\mathbf {A}}{\mathbf {x}} \:\:\:\:\: \text{[Symmetric } A\text{]}\]
- Identities
- The Product Rule:
- \[{\displaystyle {\begin{aligned}\nabla (\mathbf {A} \cdot \mathbf {B} )&=(\mathbf {A} \cdot \nabla )\mathbf {B} +(\mathbf {B} \cdot \nabla )\mathbf {A} +\mathbf {A} \times (\nabla \times \mathbf {B} )+\mathbf {B} \times (\nabla \times \mathbf {A} )\\&=\mathbf {J} _{\mathbf {A} }^{\mathrm {T} }\mathbf {B} +\mathbf {J}_{\mathbf {B} }^{\mathrm {T} }\mathbf {A} \\&=\nabla \mathbf {A} \cdot \mathbf {B} +\nabla \mathbf {B} \cdot \mathbf {A} \ \end{aligned}}}\\ \implies \\ \nabla (fg) = (f')^T g + (g')^T f\]
- Thus, we set our function \(h(x) = \langle f(x), g(x) \rangle = f(x)^T g(x)\); then,
- \[\nabla h(x) = f'(x)^T g(x) + g'(x)^T f(x).\]
- The answer to the product rule is as follows:
- \(\nabla (f^Tg) = f'g + g'f\)
or - \[\nabla (x^TAx) = \nabla (x)^T(Ax) = (x)'(Ax) + (Ax)'x = Ax + A^Tx = (A+A^T)x\]
Challenges in Machine Learning
-
The Curse of Dimensionality:
It is a phenomena where many machine learning problems become exceedingly difficult when the number of dimensions in the data is high.- The number of possible distinct configurations of a set of variables increases exponentially as the number of variables increases:
- Statistical Challenge: the number of possible configurations of \(x\) is much larger than the number of training examples
- Further Reading (Concepts)
- Yoshua Bengio on Curse of Dimensionality (Quora)
- The number of possible distinct configurations of a set of variables increases exponentially as the number of variables increases:
- Local Constancy and Smoothness Regularization:
Prior believes about the particular data-set/learning-problem can be incorporated as:- Beliefs about the distribution of parameters
- Beliefs about the properties of the estimating function
Expressed implicitly by choosing algorithms that are biased toward choosing some class of functions over another, even though these biases may not be expressed (or even be possible to express) in terms of a probability distribution representing our degree of belief in various functions.
The Local Constancy (Smoothness) Prior: states that the function we learn should not change very much within a small region.
Mathematically, traditional ML methods are designed to encourage the learning process to learn a function \(f^\ast\) that satisfies the condition:
\(\:\:\:\:\:\:\:\) \(\:\:\:\:\:\:\:\) \(f^{*}(\boldsymbol{x}) \approx f^{*}(\boldsymbol{x}+\epsilon)\)
for most configurations \(x\) and small change \(\epsilon\).A Local Kernel can be thought of as a similarity function that performs template matching, by measuring how closely a test example \(x\) resembles each training example \(x^{(i)}\).
Much of the modern motivation for Deep Learning is derived from studying the limitations of local template matching and how deep models are able to succeed in cases where local template matching fails (Bengio et al., 2006b).Decision trees also suffer from the limitations of exclusively smoothness-based learning, because they break the input space into as many regions as there are leaves and use a separate parameter (or sometimes many parameters for extensions of decision trees) in each region. If the target function requires a tree with at least \(n\) leaves to be represented accurately, then at least \(n\) training examples are required to fit the tree. A multiple of \(n\) is needed to achieve some level of statistical confidence in the predicted output.
In general, to distinguish \(\mathcal{O}(k)\) regions in input space, all these methods require \(\mathcal{O}(k)\) examples. Typically there are \(\mathcal{O}(k)\) parameters, with \(\mathcal{O}(1)\) parameters associated with each of the \(\mathcal{O}(k)\) regions.
Key Takeaways:
- Is there a way to represent a complex function that has many more regions to be distinguished than the number of training examples?
Clearly, assuming only smoothness of the underlying function will not allow a learner to do that.
The smoothness assumption and the associated nonparametric learning algorithms work extremely well as long as there are enough examples for the learning algorithm to observe high points on most peaks and low points on most valleys of the true underlying function to be learned. - Is it possible to represent a complicated function efficiently? and if it is complicated, Is it possible for the estimated function to generalize well to new inputs?
Yes.
The key insight is that a very large number of regions, such as \(\mathcal{O}(2^k)\), can be defined with \(\mathcal{O}(k)\) examples, so long as we introduce some dependencies between the regions through additional assumptions about the underlying data-generating distribution.
In this way, we can actually generalize non-locally (Bengio and Monperrus, 2005; Bengio et al., 2006c).
Many different deep learning algorithms provide implicit or explicit assumptions that are reasonable for a broad range of AI tasks in order to capture these advantages.-
- I mean that the algorithm should be able to provide good generalizations even for inputs that are far from those it has seen during training. It should be able to generalize to new combinations of the underlying concepts that explain the data. Nearest-neighbor methods and related ones like kernel SVMs and decision trees can only generalize in some neighborhood around the training examples, in a way that is simple (like linear interpolation or linear extrapolation). Because the number of possible configurations of the underlying concepts that explain the data is exponentially large, this kind of generalization is good but not sufficient at all. Non-local generalization refers to the ability to generalize to a huge space of possible configurations of the underlying causes of the data, potentially very far from the observed data, going beyond linear combinations of training examples that have been seen in the neighborhood of the given input.
- Non-Local VS Local Generalization correspond to Distributed VS Local Representations
-
- Deep Learning VS Machine Learning:
The core idea in deep learning is that we assume that the data was generated by the composition of factors, or features, potentially at multiple levels in a hierarchy.
These apparently mild assumptions allow an exponential gain in the relationship between the number of examples and the number of regions that can be distinguished.
The exponential advantages conferred by the use of deep distributed representations counter the exponential challenges posed by the curse of dimensionality.
(Many other similarly generic assumptions can further improve deep learning algorithms).- Further Reading: (on the exponential gain) Sections: 6.4.1, 15.4 and 15.5.
-
Manifold Learning:
A Manifold - a connected region - is a set of points associated with a neighborhood around each point. From any given point, the manifold locally appears to be a Euclidean space.
Manifolds in ML:
In ML, the term is used loosely to designate a connected set of points that can be approximated well by considering only a small number of degrees of freedom, or dimensions, embedded in a higher-dimensional space. Each dimension corresponds to a local direction of variation.
In the context of machine learning, we allow the dimensionality of the manifold to vary from one point to another. This often happens when a manifold intersects itself. For example, a figure eight is a manifold that has a single dimension in most places but two dimensions at the intersection at the center.
Manifold Assumptions:
Many machine learning problems seem hopeless if we expect the machine learning algorithm to learn functions with interesting variations across all of \(\mathbb{R}^n\). Manifold learning algorithms surmount this obstacle by assuming that most of \(\mathbb{R}^n\) consists of invalid inputs, and that interesting inputs occur only a long a collection of manifolds containing a small subset of points, with interesting variations in the output of the learned function occurring only along directions that lie on the manifold, or with interesting variations happening only when we move from one manifold to another. Manifold learning was introduced in the case of continuous-valued data and in the unsupervised learning setting, although this probability concentration idea can be generalized to both discrete data and the supervised learning setting: the key assumption remains that probability mass is highly concentrated.
We assume that the data lies along a low-dimensional manifold:- May not always be correct or useful
- In the context of AI tasks (e.g. processing images, sounds, or text): At least approximately correct.
To show that is true we need to argue two points:- The probability distribution over images, text strings, and sounds that occur in real life is highly concentrated.
Uniform noise essentially never resembles structured inputs from these domains.
- We must, also, establish that the examples we encounter are connected to each other by other examples, with each example surrounded by other highly similar examples that can be reached by applying transformations to traverse the manifold:
Informally, we can imagine such neighborhoods and transformations:
In the case of images, we can think of many possible transformations that allow us to trace out a manifold in image space: we can gradually dim or brighten the lights, gradually move or rotate objects in the image, gradually alter the colors on the surfaces of objects, and so forth.Multiple manifolds are likely involved in most applications. For example,the manifold of human face images may not be connected to the manifold of cat face images.
Rigorous Results: (Cayton, 2005; Narayanan and Mitter,2010; Schölkopf et al., 1998; Roweis and Saul, 2000; Tenenbaum et al., 2000; Brand,2003; Belkin and Niyogi, 2003; Donoho and Grimes, 2003; Weinberger and Saul,2004)
- The probability distribution over images, text strings, and sounds that occur in real life is highly concentrated.
Benefits:
When the data lies on a low-dimensional manifold, it can be most natural for machine learning algorithms to represent the data in terms of coordinates on the manifold, rather than in terms of coordinates in \(\mathbb{R}^n\).
E.g. In everyday life, we can think of roads as 1-D manifolds embedded in 3-D space. We give directions to specific addresses in terms of address numbers along these 1-D roads, not in terms of coordinates in 3-D space.Learning Manifold Structure: figure 20.6
-
In other words, \(p_{\text{model}}(x ; \boldsymbol{\theta})\) maps any configuration \(x\) to a real number estimating the true probability \(p_{\text{data}}(x)\). ↩
-
Ideally, we would like to match the true data-generating distribution \(p_{\text{ data }}\), but we have no direct access to this distribution. ↩
-
The perspective of maximum likelihood as minimum KL divergence becomes helpful in this case because the KL divergence has a known minimum value of zero. The negative log-likelihood can actually become negative when \(x\) is real-valued. ↩
-
Statistical efficiency (measured by the MSE between the estimated and true parameter) is typically studied in the parametric case (as in linear regression), where our goal is to estimate the value of a parameter (assuming it is possible to identify the true parameter), not the value of a function. ↩


title = "Partners"
url = "/partners"
layout = "default"
is_hidden = 0
[ourteam]
team = 5
results = 100
sortOrder = "name asc"
[ourteam ourteam2]
team = 6
results = 100
sortOrder = "name asc"
[ourteam ourteam3]
team = 7
results = 100
sortOrder = "name asc"
==


This structure contains information required to configure an address translation region. More...
#include <csl_rat.h>
Data Fields | |
| uint64_t | sizeInBytes |
| uint32_t | baseAddress |
| uint64_t | translatedAddress |
This structure contains information required to configure an address translation region.
| uint32_t CSL_RatTranslationCfgInfo::baseAddress |
The 32-bit base address used in the address comparison
| uint64_t CSL_RatTranslationCfgInfo::sizeInBytes |
The size of the region (in bytes). If this value is not a power-of-2, it is automatically rounded-up to the next power-of-2 size.
| uint64_t CSL_RatTranslationCfgInfo::translatedAddress |
The translated address returned on a successful match


Background
Last year, I spent most of my free time becoming familiar with Ruby, Rails, and Scala. I’m by no means an expert, but I’m trying to follow the advice of the Pragmatic Programmers and learn a new language/system every year. This year, it’s going to be the iPhone API and Objective-C. I coded some WebObjects applications in Objective-C many years ago, so the language isn’t totally unfamiliar, but the iPhone SDK, obviously, is. After working my way through the first several chapters of the Dudney and Adamson’s “iPhone SDK Development” book, I decided I was ready to try my first real app.
First App
My girlfriend and I enjoy wine, but are largely novices. On a trip to Napa we started writing down the wines we’d tried and keeping personal ratings of them. The idea being twofold: try to remember good wines/avoid bad ones, and try to hone in on what our tastes are. There are so many wines and so many flavors and smells, it’s just impossible to keep in your head. So, I created a Google Spreadsheet and a form to allow us to enter wines on the go, via our smart phones. The Goolge Docs form is less than usable, and this seems like the perfect job for a native iPhone app.
User Experience
Aside from learning the ins and outs of the SDK, another main goal is focus on making a highly-usable application (at least for myself and my girlfriend). So, before diving into Core Data and modelling the domain, my first version of the applicaton simply presents the “add a new wine” screen. This will be the screen I spend the most time in, and it’s important to make it fast and easy.
The Google Spreadsheet currently collects:
- Wine Name
- Vineyard
- Vintage (year)
- Varietal
- Type (e.g. red/white)
- Where we had
- When we had
- My rating
- Her rating
- Tasting Notes
That’s obviously too much for one iphone screen, and, if I’m entering something in a hurry, I might not have time to go through all that. Further, I think adding multiple ratings makes things a bit confusing, so I focussed my screen on the identifying characteristics (name/vineyard/vintage), the rating, and the type:
As you can see, I tried to fit the “where” on the bottom, as well as a “enter more information” button. After loading this on the phone, it was just not working out, plus I was missing the varietal, which is fairly important (at least to me).
My next attempt simply ignores the “add more” concept, and, by using the segmented control, I was able to fit the type into a more compact area, while including the varietal as a “slot-machine”/picker.
My plan is to load this onto the phone and try using it the next few times I’m out. I figure I can do without the tasting notes or location if it keeps the UI simple
The Code
Objective-C and XCode are a whole different world from Java or Ruby. First thing that bit me is the memory management. I was familiar
with the rules (or so I thought). Basically, if you call a method like “copyXXX” or “initXXX” to create a new object, you must
release
it when you are done with it. If you get access to an object through any other means, you should not release it.
// my class has an NSArray called varietals that I'm initializing here
varietals = [NSArray arrayWithObjects:
@"Barbera",
@"Cabernet Franc",
// snip
@"Zinfandel",
@"Other",
nil];
Since I’m not calling an “init” style method, I figured I didn’t need to release it in dealloc
. For some reason, this made me decide that I didn’t need to retain
it. This caused my first crash and trip through the debugger to figure out what in the heck is going on.
// my class has an NSArray called varietals that I'm initializing here
varietals = [[NSArray arrayWithObjects:
@"Barbera",
@"Cabernet Franc",
// snip
@"Zinfandel",
@"Other",
nil] retain];
Since I needed varietals
to survive the method where I was initializing it, I had to retain
it.
Other struggles
Interestingly, laying out the UI and getting that hooked up isn’t too bad; the thing I’m having the most trouble with is figuring out what is the “unsurprising” way of doing things; tutorial code from books is rarely very well written; it’s usually done in a certain way to make concepts clear, but I haven’t yet found a definitive “here’s how you organize your code” or “iPhone best practices” that seems relatively comprehensive.
At any rate, I figure if I can get the UX to be reasonably good, the rest should sort itself out.
I’ll continue posting about my progress here on my blog. If I can get the app polished and working, my girlfriend might demand and Android version. Would be interesting to compare the two paradigms (especially since Android is all Java, which has been my bread-and-butter for many years).
Continued in Part 2, Part 3, Part 4, and Part 5