Cyber Security Fundamentals
Lecture 1 - Introduction
- Security State/Condition of being pretected from or not exposed to danger from some internal or external threat
- Hacking against the law
- Unauthorised access to computer material
- Unauthorised access with intent to commit or facilitate a crime
- Unauthorised modification of computer material.
- Making, supplying or obtaining anything which can be used in computer misuse offences.
”..prevention and detection of unauthorised actions by users of a computer system” - Deiter Gollman
“the prevention and detection of unauthorised actions by users of a computer” - Microsoft
Socio-technical Systems
-
System is a purposefull collection of interrelated ocmponents of differen kinds, which work together to achive some ovjective
- Socio-Technical systems include technical ystems,but also organisational processses and people who use and interact with the technical sustem
- governed b organisational and regulatory policies and rules
- Computer Security - protection of hardware, software & data assets
- Information Security - protection of information and information systems from unauthorized access, use, disclosure, disruption, modification, or destruction in order to provide confidentiality, integrity, and availability
Security model
CIA
-
Confidentiality - Prevention of unauthorised disclosure
-
Integrity - Prevention of unauthorised modification of information
-
Availability - Prevention of unauthorised withholding of information and rescources
- Vulnerability - Weakness in your system
- Hardware
- Software
- Network
- Personnel
- Location
- Organisational
-
Threat - Potential exploitation of vulnerability
-
Attack - Attempted violation of vulnerability
-
Passive - Just watches and maybe collect data
- Active - Attempts to change resources or operations
Threat Types
- Physical Damage
- Natural Events
- Loss of essential services
- Disturbances due to radiation
- Compromise of Information
- Technical Failures
- Unauthorised Access
- Compromise of Functions
Deal with threats
- Reduce/Remove threats by
- Prevent
- Detect
- React
- Correct
- Accepted (“it’s not a bug its a feature”)
- Avoided
- Transfered (give the problem to someone else)
Doing Security == Risk Management
- Asset identification
- Risk identification
- Find assest’s vulnerabilities
- Find the relavent threats
- Risk treatment analysis
-
Treat the risk
- Asset protection
- What are my assets
- What danger are they in
- How can I protect it
Section 4 - Security Engineering
-
Security Engineering is about building systems to remain dependable in the face of malice , error or mischance
- What are ny security requirements
- What are my policies
- Policy and Guidence
- Policy- What you want to achieve?
Poits of Interest
- Simple > Rich mechanisms
- Trade-offs
- cost/speed
- effort
- Should the effort I put in to securing the data be relatied to the worth of the data?
Key principles
- Easist penetration
- Adequate protection
- Weakest link
-
Effectiveness
-
Security should not be an afterthought
- Crytography is not a silver bullet
Actors
- Amatues/Script kiddies
- Hackers
- Black Hat
- White Hat
- Gray Hat
- Hactivist
- Carrer criminals
- Terrorists
- Government
Intro to Cryptography
- Confidentiality
- Public key encryption
- Block ciphers
- Stream ciphers
- Integrity
- Cryptograhic hash function
- Message authentication codes
- Authenticity
- Digital signatures
Primitives
Hashfunctions/Message digiest
- Takes some data in
- Gets signiture out
Funcion to compute a unique random signature for some data
- Guarantee towards: Data Integrity
- Preimage resistance
- Second Preimage resistance
- Collision resistance
- Eg: MD, SHA
Block Cyphers/Symmetric crytography
- Guarantee towards: Confidentiality
- Uses sameto encrypt and decrypt
- Efficient for large messages
- Eg: Blowfish, TripleDES, Skipjack, AES
Asymmetric ciphers
- 3 functions
- 1 to Encrypt
- 1 to Decrypt
- 1 to create key
- Guarantee towards: Confidentiality
- Key Pairs
- Encrypting and signing
- Inefficient on large data
- Eg: DSA, (EC)DSA, ECC, RSA, ElGamal
Workflows
- Primitives can be used to have guarantees for more than one of
- Information Security
- Efficient Information Security
- Sender Authentication
- Secrecy with Authentication
- Secrecy with Signature
- Secrecy with Integrity
- Signature with Appendix
- Secrecy with Signature with Appendix
Digital Signitures
- Authenticity of message origin
- Non-Repudiation of message origin
- Message Integrity
Public Key Encryption & Digital Signitures (Combination)
- Message confidentiality & integrty
- Authenticity of message origin
- Non-Repudiation of message origin
KEM/DEM (Hybrid)
- Symmetric Encryption to encrypt data
- Assymetric to encrypt symmetric key
Implementing
- RSA System
- Encryption
- Digital signitures
- Descrete logarithm system
- Key exchange
- Digital signiture algorithm
- Discrete logarithm integrated encryption scheme
Lecture 3 - Introduction Cryptography 2
- Why love crypto?
- Provides mathematical guarantee
- CIA
Block Cipher
- symmetric ciphers
- same key to encrypt & decrypt
- Bitwise operations on fixed size blocks of data
- XOR with key
- initialisation vectors
- take data
- keep encrypting until unreadable
- fast
- easy to implement in hardware
- 2 families
- feistel, IDEA
DES
- Project Lucifer
- several transforamtions repeated one after the other
- sub keys generated from one main key
- Insecure Algorithm
- NSA reduced key-length to 56 bits
- This made it easier to bruteforce
Triple DES
- use DES 3 times
- encrypt wiht 1 key
- decrypt with 2nd key
- encrypt with 3rd key
- noone uses this anymore
- if use same key there are backwards compatability
-
dont use this
- is it just XOR? no
AES
- symmetric crypto is inherantly insecure
- DES is insecure
-
replaced by AES
- Rijndael won in a competition
Rijndael
- Based on permutations and substitutions
- Block and Key size
- 128, 192, 256, 512
- variable number of rounds dependent on key size
- Fast
- implemented in hardware
- currently no serious weakness
Modes of Operation
- ECB
-
ECB can leave plaintext data patterns
- break into blocks
-
encrypt and concatinate back
- operating on each chunck individually
-
no block big enough for the wholemessage
- problem
- you dont know when the blocks end
- order of blocks?
- CBC
- ouput fed in to the next round
- creates random noise
-
operates on each block individually
- PCBC
- CFB
- OFB
- CTR
IV and Padding
- initialisation vectors
- dont have to be secret
- keep secret key
- different init block vector with diff keys
- last block padding
- ensure final block has correct length
Secret Sharing
Deffie-Hellman Key Exchange
PRNGs
- want to makesure they are truely random
- virtually impossible
- always have some sort of structure
- hidden deep inside the structure
- enough to generate sequnce from a seubsequence
- hidden deep inside the structure
- cryptographic
- cannot be predicted from some of the numbers in the sequence
Generation Strats
- Computer clocks
- Keyboard latency
- Using random noise
- Basically you mash on your keyboard and it takes bits from the clock and mash it together
Message Digests
- given any message, easy to compute the digest
- given any hash, hard to compute the message
- need to limit
- hash collision
- birthday attack given 2 random message, should not produce the same digest
Designing
- DES in secret
-
AES in open competitions
- Kerckhoffs 2 Principle
- security in your algorithms should be in your operations and in your keys
- Linus law
- many eyes, catch many bugs
- NOTHING IS SECURE
Lecture 4 - Cryptography in the wild
Applied
- Primatives & Schemes not enough for security
-
Key are just numbers
- Algorithms
- Which one to use?
Data at rest is data that doesn’t “move”
- Can be solved by crypto
- KEM/DEM
- Public Key Encryption, Symmetric Encryption…
- Key management is the real issue
- Public Key Infrastructure: Centralised/Decentralised
- Expensiveness of Encryption
- perfect forward secrecy
- end to end encryption
- additional mechanisms to manage permissions
Data in flight is data beign moved from one domain to another
-
Securing data in flight
- Crypto to construct secure channels
- End-2-End Encryption
- what level of the osi am i opperating at
- Network Layer has IPSec
- Transport Layer has TLS
- Application signal, Cryptocat, OTR…
- Keys?
- Centralised/Decentralised
DONT IMPLEMENT OR DESIGN YOUR OWN CRYPTO. unless you are a trained engineer
-
Theroetically secure != Implementation secure
-
Use good standard libraries
- you have to look at domain, cost and effort keylength.com
ISO 27K
- Guidence on how to do things
- Regulation of Cryptographic Controls
- what you need to be aware of when implementing
- are there import/export restrctions
- are there restrictions on using encryptions
- law enforced access to encrypted info
- get legal advice
- Policies
- where we are we going to use it
- impact of policy
- roles and responsibilities
- management approach
- mobile data
-
key management
- complience
- national and international Laws
- decrees
- industrial regulations
- codes of conduct
- stakeholder expectations
- contracts/agreements
- best practices
- company specific policies and standards & guidelines
- regulations/administratives orders
- Laws
- EU Laws
- British Laws
Advanced
- Encryption Schemes
- Signature Schemes
- Secret Sharing Schemes
-
Provable Crytographic
-
how are these used
- Authentication protocols
- Anonymous Protocols
- eVoting
- Multi-Party Computation
- Distributed Authorisation
- Searching Encrypted Data
Attacking
- if you want to attack the mathematics of encryptions, get a PHD
- factorising composite numbers
- taking discrete logarithms
Attack the standard
- make sure the system has a backdoor
Attacking the System
- Attackers cna
- evesdrop
- insert message
- impoersonate
- hijack
- DOS
Human Factors
- Espionage
- send your agents
- rubber hose cryptoanalysis
- how can you do rubber-hose resistant crypto?
- Token Attacks
Code Vulnerability
- just leave it to the experts
- dont make your own
Side channel attacks
- Differential Power Analaysis
- estimate key lengths from power consumption of computations
- Timing attacks
- find timings for computatioins
- Electromagnetic attacks
- analasys the electromagnetic radiation
- Ghost data
- non-deletion of sensitive data
- cold boot attack
- Differential Fault Analaysis
- inducing faults in operation
- Acoustic cryptanalysis
- listening to CPU output for computation noise
- most of these are defeated by noise
Tutorial
- a
- DAE is open, DES is closed. open i sbetter . AES is more cheaper. des on its own is insecure.
- tuxPenguin, cbc has the chain,
- there is a small amount of digests to cycle through
- s
- they are different algs, diff fundamentals, so cost/efort to attack is different
- encrypt message with private key, exchange, encrypt wih private key, exchange. not vulnerable to MITM attack, no way to make sure who you are talking to s who you are talking to
- not possible to get truely random number, psudo there is structure, crypto, hard to determine what it means
- ifyou know theseed you can guess the random number.
Lecture 4 - Authentication
- Enrollment : Establish Credentials
- Challenge & Response : Check validity of credentials
- Styles
- Direct/Brokered
Local
- Commonly used: (Identity * Password)
- Store salted digest of passowrd for security
- MD5, SHA
- when logging in
- password salted & hashed
- compared with stored hash
Dictionary Attacks
- attackers can calculate variations of a password and form salts hashes
- if there is a match, we found the password that is begin used
- david kliein
- gets usernames, initias, login name
- female and male names
- places, names of famous people, numbers, vulgar phrases, keyboard patterns
- created permutations of above
- capitalisations from previous lists
Salting
- a random string or value that is added to password before encryption
- add randomness to password so its harder to guess
- large salts are good
- salt string is publicly vailable, each password has individual salt
- dont store password as you are given, add some randomness
- keep password securely
password complexity
- UPPER & lower case
- numbers
- don’t common words
- use special characters
-
at lease X characters long
- entropy - how guessable
- can measure password strength using information theory
- how much information is carried in a single characters
- first character ~ 4.7bits
- 2-8 ~2.3bits per character
- 9=> ~1.5bits per character
Passphrases
- trend
- long and easy to remmeber
- depending on what device you use, use different passowrd
Remote
-
What is a secure channel
- there might be someone evesdroppting
- impersonating the sendee
- they can manipulate it
- how do we talk securely?
- we can use cryptography
- use secrecy with signature with appendix using KEM/DEM
- use symmetruc crypto & deffie-hellman
- how to prevent replay attackers?
- ensure message freshness by adding
- random info
- identity & timestamps
- ensure message freshness by adding
Brokered Authentication
- have a trusted third Party
- trusting thirdparty for keys
- we are assuming he is good
- we are trusting him to be honest
- he might be malicious
Needham-Schroeder Protocols
- based off deffie Hellman
-
trusted key server that generates session keys
-
symmetric & asymemetric public key
-
both symmetric and assymetric are suscepible to replay attackers
- flaw
- ryan can get alice’s id and send it to ken
- solution
- add ID to the message
- if ID changes, we know there is a man-in-the-middle
Kerberos
User Authentication in a Distributed systems
- User based client auth
- client-led auth
- mutual auth
Kerberos
- 3 headed dog of secure authenticated netowrk communication
- each network authenticated
- each message authenticated
-
each message encrypted
- requires
- authentication service
- ticket granting service
overview
- sign to service
- alice signs in with password used to create shared secret
- create a ticket to authorise with ticket services
- ticket contains who it was generated for, and a key to enable communication with ticket service
- restricted lifetime to prevent replay attacks
- get permission to talk to bob
- alice asks ticket service for ticket to enable communcation with bob
- T~ttl is proof that Alice can use TGS
- timestamp is for freshness
- TGS sends somthing for Bob
- Talk to bob
- alice uses ticket to talk to bob
- Alice is sending Bob information on how to talk that could have only been created by TGS
- Modification of timestamp tells Alice, that Bob is who he says he is
- Bob already has K~B, TGS and this can gain access to K~A,B
-
repeat steps 2 & 3
- Authentication Protocol based on Needham-Schroeder with fixes
- Single-Sign on
- authenticate with AS and get timed access to system
- this ticket can be used to request access to other servicies like bob
-
Simplified public key variants
- advantages
- auth on distributed system
- single-sign-on
- disadvantages
- single point of failure
- not federated
Lecture 6 - Publc Key Infrastructure
PKI
- Third party to deal with public keys
- certificate authorities binds public keys to IDs
- when regestering your pub key
- when authority generates your public/private keys
- binding is done with digital signitures
PK Certificates
- proof of identity
- forged certificates?
- attackers need the ability to sign the certificate which is hard for them to do
- fake certificates?
- Authority must berify the owner’s ID before signing trhe certificate
- Need certificate authority’s verifying key to check authenticity
Key Management
- certificate issuing and publication
- certificate revocation
- compromised keys
- identity change
- termination of membership
- key pair Generation
- private key escrow, backup, arhiving, and recovery
- public key archiving
- update
- logging of actions
Certificate Chains
- problem
- who trusts the trusters…
- how to balance certificate creation
- multiple certificate authorities at different levels
- answer
- each authority verification key is signing by more trustworth body
- Shape of chain
- more trustwrth keys can be embedded in hardware
- so they are tamper resistant/tamper evident
Tutorial
- w
- a
- a
- how you deal with enrolment.
- easy to remmber, more entropy so its harder to guess, easier to type on a different device like a mobile. easier to be attacked by over-the-shoulder attacks.
- a
- Password crackers
- a
- a
- a
Web security
How does it work
- browser ask webserver for a website which might be connected to a backend
- the website is executed on the client
HTTP
- used in most transmition
- stateless
- easier to implement
- must provide id with every transmission
- no built in encryptions
GET
- retrive index.hmtl
- asks webserver for specific resource
- answer - get the resource you want
- should only be used for info retreival
- can have variables which is sent to webserver
GET /index.html HTTP/1.1
GET /index.html?foo=bar HTTP/1.1
POST
- used to send data and retrive changed resources
- sending credentials is worst way to use it
POST /index.html HTTP/1.1
Host:example.com
Content-type:application/x-www-form-ulencoded
Content-length: 7
foo=bar
Cookies
- data that a webserver can store in a client
- like session id
- should be complex
- shouln’t be easy to guess
- mainly used to make sure that two request came from the same client
Setting HTTP Cookies
HTTP/1.1 200 OK
Content-type: text/html
Set-Cookie: foo=bar
...
Using HTTP Cookies
Get /index.html HTTP/1.1
Host: example.com
Cookie: foo=bar
OWASP Top 10 Applicatoin Security Risk 2017
- Injection
- send data to a server that doesn’t chack the data
- then used in a interpreter
- Main problem
- user provided data is used without sufficient checks
- eg SQL Injection
- results in
- stolen user data, creditcard Data
- DOS
- take over host
- broken authentication
- able to bruteforce. since a lot of people use weak passwords
- Main problem
- incorrect implementation/lack of knowledge
- eg login page bruteforce
- results in
- compromise of systems
- money laundering
- unauth info disclosure
- sensitive data exposure
- web apps and apis not protecting sensitive properly
- be careful how you save your Data
- main problem
- not putting enough thought/energy into data protection
- eg. using automatic database encryption
- results in
- offences againt priv Laws
- loss of trust
- XXE
- a way to retrive files that are not retriveable
- Main problem
- trusting external prvided inpout
- eg
<? xml version=“1.0” encoding=“ISO-8859-1”?> <!DOCTYPE foo { <!ELEMENT foo ANY > <!ENTITY xxe SYSTEM “file:///etc/passwd”>]> <foo>&xxe;</foo>
- results in
- extract databaseinfo gathering
- broken access control
- ability to switch to more privlaged user/access to privlageed user
- Main problem
- trusting external provided input
- eg
- SQL Inj
- results in
- privelage escalation
- security misconfig
- insecure default configs
- Main problem
- Lack of knowledge
- eg
- enabling directory listing
- results in
- unauth Access
- sys takeover
- info gathering
- XSS
- code injection
- Main problem
- user provided data is used witout sufficient checks
- results in
- data loss
- insecure deserialization
- when an app receives hostile serialized objects
- influence control flow
- remote code execution
- Main problem
- accept serialised objects from instusted sources or user provided data without sufficient checks
- eg.
- supper cookie containing id, passoword, hashed
- results in
- attacker can cahgne those values
- privilae escalation
- remote code execution
- using components with known vulnerabilities
- Main problem
- lack of knowledge
- dont ignore software update notifications
- results in
- privlaged excalation
- data loss
- remote code execution
- Main problem
- insufficient logging & monitoring
- Main problem
- without identifying an attack you can’t act
- eg
- vulnerabuility scanning
- bruteforce password attackers
- results in
- being succussfully attacked
- Main problem
Websecurity 2
Injection
- SQL Injection
$user = $_POST['user']; $pass = $_POST['passoword']; $query = "select * from u_list where u_name = '$user' and password = '$pass';" ... if ($result > 0) {// user login successful
-
what could an attacker submit in order to gain access?
- Command Injection
$cmd = "python get_data.py".$_GET['sort']; $list = system($cmd);
- blind sqlinjection
- true or false Injection
- more difficult but can lead to same Effectiveness
- prevention
NEVER TRUST USER PROVIDED INPUT
- use functions to escape inputs
- only use frameworks that are considered safe
- use
LIMIT
Broken authentication
- weak passwords
- one password for all == single point of failure
- use multi factor authenticator
-
password rotation
- weak passowrd storage
- plain plaintext
- unsalted hashes
- weak hashes
- prevention
- use moden hash function
- check for weak passwords
- passwords at least 8 char long
- harden against account enumeration attack
- multifactor authentication
- log auth failure and alert admins
broken access control
- prevention
- don’t let attacker modify the checks or metadata
- disable directory listing
- log access control failures and trigger alerts
XSS
- Prevention
DONT TRUST USER INPUT FFS
- use frameworks that automatically escapes this for you
- escape data
Tutorial answers in moodle
Lecture 9 - Network Security
Important Nework Protocols
Data Link Layer IEEE 802.3
- important information in an ethernet frame
- destination mac address
- source mac address
- payload
- checksum
- MAC address
- 48bit
- first 24bit organisationally unique (OUI)
- rest of 24bit identifying NIC specific
- special addresses
- broadcasters
- multicast
Data Link Layer IEEE 802.11
- MAC for wifi
- adds extra frame types
- Management frames
- Control frames
- Data frames
- MAC address of the Access Point(AP) is added to the header
Network Layer
IPv4
- used for addressing
- 32bit
- private IP ranges;
- <IP_ADDRESS> to <IP_ADDRESS>
- <IP_ADDRESS> to <IP_ADDRESS>
- <IP_ADDRESS> to <IP_ADDRESS>
- important info
- source address
- destination address
IPv6
- 128 bit
- 2^64 addresses
- atateless auto config
Data Link Layer: ARP
- Address Resolution Protocol
- asks for MAC address for an IP adress
- also used for duplicated IP addresses
Transport Layer
- ports
- 65532 ports
- well known ports < 1024
- admin rights to open those ports
- rest can open without permission (in linux at least)
- additional registered ports
-
direct link between port and service
- UDP
- connectionless
- fire and forget
- transmission not ensured
- connectionless
- TCP
- 3-way handshake
- SYN (seq = x)
- SYN (seq = y); ACK x + 1
- ACK y + 1
- ensures order (seq)
- reliable data stream
- 3-way handshake
Application Layer
DNS
- Domain Name System
- translates domain name to IP address
Network Structure
Wired
Wireless (WiFi)
Interconnected
Attack Vectors
- clients
- IP phones
- server
- switches/access points
- router
- ……
DOS
- overload the target with requests
- eg
- SYN-Flood
- TCP-Queue overflow results in DoS
DDOS
- eg
- problem: target might have too much POWAAH
- solution: use multiple attackers
- fun with more friends
- minimum requirements: friends
ARP Flooding
- switch
- takes MAC frame
- checks destination MAC
- looks up port number in an internal MAC lookup table
- forward the frame
MAC Flooding
- switch gets the lookup table from
- observing traffic on physical ports
- extracting the source MAC address of connected port
- storing it
- problem
- storage not infinite
- exploit
- send a lot of frames with different soure addresses
- when storage space runs out
- switch becomes a hub
- gain?
- hub broadcasts a frame to all ports
- allows monitoring of all netowrk traffic
- reduce performance
- hub broadcasts a frame to all ports
ARP Poisoning
- all clients store lookup tables
- ip to MAC
- arp helps to build this table
- this table’s validity is not infinite
- client asks who asks this ip
- server replies “ME”
- attacker replies “MEMEMEMEMEMEMEMMEM”
- yelling as loud as possible helps
- gain?
- all traffic redirected to attacker
-
man-in-the-middle
Lecture 10 - Network Security 2
Attack Vendors
Fun with IPv6
- problem
- auto configs enabled the IPv6 traffic
- windows pref ipv6 > ipv4
- without any settings
- enabled by default
- MitM for all the networks traffic
- Defence
- disable IPv6
- later on patched
Social Engineering
- attaks users that aren’t educated
- Factors
- Technical
- power plugs
- ethernet sockets
- Human
- alertness/awareness of personal
- knowledge
- continuous education of staff
- Technical
- Defence
- Technical
- block unused ports at the switch
- human
- keep awareness high
- inform staff about methods
- keep it non technical
- Technical
DNS Tunneling
- send data to a blocked target
- control a DNS Server
-
ability to send DNS lookup
- domain names <= 253 characters
- with label <= 63 characters
- messages base64 encoded
- make sure you dont look up the same domain name twice
Defence
Firewall
- strong defence
- still trafic needs to come in and output
- blocking everything is not an option
- blocking everything is generally a good start
- white list vs black list
- descisions beased on rule sets
- rules can be classified into categories
- prerouting
- all acitons before a routing decision is made
- input
- actions on incoming traffic
- outout
- actions on outgoing traffic
- postrouting
- all actions after a routing decision is made
- prerouting
- everytime you write down firewall rules
- it should be precise as possible
src/dst ip
src/dst port
protocol
- don’t permit everything
- only permit necessary ports
- it should be precise as possible
Proxy Server
- used to control the flow from the inside to the outside
- can specify all traffic to go through proxy server
- can filter traffic and block traffic
- can monitor traffic
-
perfomance improvements
- different types of proxys
- HTTP
- SOCKS
- all kinds of traffic
- TOR (The Onion Router)
- …
- ssh socks proxy tunnel
- traffic from lan to ssh is encrypted
IDS
- monitoring traffic
- filter malicious traffic
- signature based
- anomaly detection (ML based)
- alarm admin if malicious traffic is found
IPS
- same as IDS but
- can take actions automatically
- blocks ports, IP addresses …
- can take actions automatically
VPN
- connects to a network through another one
- connects to private network via public infrastructure
- secure connection via insecure channels
- access to all private network’s resources
- can be used to circumvent restrictions
WiFi
- no phsicall security (no cables)
- potentially everybody can listen
- jamming
- deauthentication spoofing
Open Wifi
- everyone can listening
- never use unencrypted services to transmit any important piece of information
Encrypted WiFi
Wired Equivelent Privacy
- sniff as much packets as you can
- passive Attacks
- analyse the traffic
- Active
- inject the traffic
- outocme: key
- never use it
WPA2
- per packet keys
- protected against replay attacks until octber 17
- problem
- KRACK recontructs the key thorugh replay attacks
- weak per-shared keys, allow brute force
- management frames unencrypted
- deauthentication flooding
- jamming
- medium accessible for every sender


"Gameplay comes first!" aka FPS Golder Rules inspired by Smashbros & more [promoted]
pavla writes:
IN FIRST-PERSON GAMES, GAMEPLAY COMES FIRST
While we want our animations to look cool, it’s more important to make sure that they communicate the action without getting in the way of the player.
A short, snappy, but otherwise boring animation can actually be better than a fancy one with lots of anticipation and flourish because your player is going to be watching it hundreds - if not thousands of times.
It might be fun the first couple of times, but after that, the distraction or extra time it takes can be frustrating.
That said, we want to try to use the limited time we have and make things feel fluid, natural and fun. Just be aware that it’s a secondary goal.


---
group: starter-template-general-home
layout: starter-template/general/light
title: Home 5
description: Start a new project with 50+ choices of clean and professional starter templates
---
Technoart
It all started in a garage
Testimonials
Clean & Professional
People are more likely to do business with a website that looks professional, and you can achieve this with a clean & responsive design from Tecnoart.
We understand the intricate workings behind a successful online presence, and therefore provide our clients with a comprehensive service package that goes far beyond a beautiful website.
Clean & Professional
People are more likely to do business with a website that looks professional, and you can achieve this with a clean & responsive design from Tecnoart.
12+
✓ 24/7 support
✓ Unlimited revision
✓ 1 year maintenance
✓ Critical updates
All in One
For the ultimate css framework that can be used for almost any web design without custom css, look no further than the Technoart.
12+
✓ 24/7 support
✓ Unlimited revision
✓ 1 year maintenance
✓ Critical updates
Rich Example
Find inspiration in the starter template by exploring exactly what you’re looking for. Brought to you by the most generous Technoart team.


Notebooks
Story: A story about another Star
A story about another Star
Title
A story about another Star
Collection
Summary
Various statements are given with regard to the nature and movements of the celestial bodies. The first of these treats of the Moon and Stars, ending with the 'Bushman–rice' Star Canopus (B I. 291-294).  This is because they are the Stars.
Comments
1) This story is found in Book II-2
Contributors
Date
1871
Categories
Celestial bodies and aeroscopy
Keywords
Star (mother) , Star (story of) , Star (wife) , Star (husband) , Star (child) , Star (that is red) , Star (marriage of) , Star (colour of) , marriage (of the Stars) , child (of Star)
Story Pages
294-295


2 references to Windows
System.Management.Automation (2)
CoreCLR\CorePsPlatform.cs (2)
169
int result =
Windows
.NativeMethods.CoInitializeEx(IntPtr.Zero, COINIT_APARTMENTTHREADED);
175
Windows
.NativeMethods.CoUninitialize();


Which parts of your body were affected by the accident
List your injuries. This information will then be confirmed with a medical professional at a later date.
List your injuries. This information will then be confirmed with a medical professional at a later date.


### Interactivity In order to interact with the **2D stage**, you can: - `Alt+MouseWheel` to zoom in/out - `Alt+MouseDrag` to pan the view Specific Brain stuff interaction: - `Shift+MouseOver` to show tooltips - `Ctrl+Click` to split border segment
y:


Aspiring software engineer - Graduated Fall 2019 from the University of Georgia with a degree in Finance and MIS. Over the past years, I've worked in operational finance-risk at a multi-regional bank and as a privacy and security consultant at a high-growth privacy startup, but I've always had a passion for coding, since I took my first java class in highschool. My life motto is "yolo" so I decided to fully jump in and immerse myself in code and am now a full-time student at Georgia Tech's coding boot camp.
In my spare time, I enjoy hanging out with my friends, family, and two cats, fishing whenever and wherever I can, going to the gym (though not so much anymore due to COVID), playing video games, and teaching myself other programming languages and principles.


@* * Copyright 2023 HM Revenue & Customs * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. *@ @import ssttpaffordability.model.IncomeBudgetLine @(income: Seq[IncomeBudgetLine])(implicit messages: play.api.i18n.Messages) @{income.map( budgetLine =>


Learn whatever you set your mind to.
(This project has been created for KhanAcademy in order to utilize and showcase an rollout (or infection) algorithm)
In order to use the rollout feature, please Sign in
and if you have not created an account yet, please click here to sign up.


ATLANTIS: THE LOST EMPIRE A film review by Mark R. Leeper
Capsule: While SHREK, still playing in theaters, mocks the old Disney traditions, ATLANTIS: THE LOST EMPIRE pays respectful homage to old Disney films while telling a story like H. Rider Haggard on steroids. A legendary book leads a mismatched expedition to find the mythical city still alive, though just barely, deep beneath the waves of the ocean. A little heavy on the mysticism and the fighting, this animated film is not a bad choice adventure fans. Rating: 7 (0 to 10), low +2 (-4 to +4)
In the late 1800s there was still a lot of the world that was terra incognita. Much of the map had still to be filled in and adventure stories were being written about fabulous finds of ancient cities still alive in the far corners of the world. The greatest of these stories, in my opinion at least, was H. Rider Haggard's SHE, filmed in multiple silent versions and at least two sound versions. Other authors who have written lost race stories include Edgar Rice Burroughs, Ian Cameron, and A. Merritt. What science fiction is to many branches of science and alternate history is to the study of history, lost race stories are to archeology. They are the stories of the imaginative dreams of every archeologist. (For those interested in the lost race genre I can recommend the web site http://www.violetbooks.com/lostrace-check-guide.html.)
The current film starts at a breakneck pace as a huge wave rushes foreword to engulf the advanced island-civilization of Atlantis and some strange flying machines racing it to try to save the island. Meanwhile there is something strange and mystical happening on the island, but not so powerful that it saves the island from sinking below the waves.
Flash forward to 1910 or so. Exploration runs in the family for Milo Thatch (voiced by Michael J. Fox). Thatch's grandfather searched for the lost city of Atlantis. Milo has his own ideas as to where the city can be found. His dedication and energy applied to this goal has won him a reputation of being a little demented on the subject. Then an enigmatic millionaire has his own plan to find Atlantis with the help of an ancient book thought to be lost but found by Milo's grandfather. A new expedition will search for the city and its mysterious power source. The expedition will be led by Commander Lyle Tiberius Rourke (James Garner) and begin with a descent to the ocean floor in a fabulous submarine.
The base story by Bryce and Jackie Zabel leaves room for some spectacular action scenes with an undeniable excitement. The film seems to be an H. Rider Haggard adventure in concentrated form. The story moves faster and has more action than Haggard would have given it, but the spirit is there. One thing that does seem a little out of place: most lost race stories were told in a serious tone. Because of the subject matter the stories were rarely told with much humor. There is a lot of Disney-style comedy and weird international characters on the expedition. (Is the character of Moliere based on the character of the same name in the ZBS's Ruby series? There are definite similarities.) The writers make the usual politically correct choice for the presence and ethnicity of the villain. The script in the end feels a little top-heavy with fighting and mysticism. Some may long for the subtlety of some of the writers of years past, but overall the film does have its moments.
To some extent ATLANTIS: THE LOST EMPIRE is an experiment. First it is a PG cartoon, unusual for Disney. The whimsical nature and the less realistic animation techniques seem less likely choices for the subject matter. The style might go better with a humorous animal story. Further toward the end of the film a lot of what is happening is not carefully explained and is left to the viewer's interpretation. The film does have some breathtaking images of the city Atlantis and especially of the great submarine seen all too briefly in the first part of the film. The submarine seems like a cross between Disney's Nautilus and the interior of an airship. Real submarines don't look like this, but they ought to.
Once again we have an all-star cast of voices in an animated film where their familiarity can be only a distraction. We have Michael J. Fox in the lead. We also have James Garner, Jim Verney (who died in February of 2000), Claudia Christian, Don Novello (who on frequently played the comedic Father Guido Sarducci), John Mahoney, and Leonard Nimoy as the King of Atlantis. While far from ideal, Atlantis is a good adventure film with at least some of the nostalgic feel of classic exploration films. I rate it a 7 on the 0 to 10 scale and a low +2 on the -4 to +4 scale.
Mark R. Leeper<EMAIL_ADDRESS>Copyright 2001 Mark R. Leeper
The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


| Public Properties | |
|---|---|
| CollisionLayer | Get the collision layer that this boundary is on |
| CollisionType | The collision type of the box |
| Data | The data property of the box. |
| Height | The current height of the box |
| Velocity | The current velocity vector |
| Vertices | Get the vertices array of the tlBox |
| Width | The current width of the box |
| World | Get the y world coordinate of the boundary |
| Public Constructors | |
|---|---|
| New | Constructor |
| Public Methods | |
|---|---|
| BoundingBoxOverlap | Compare this tlBox with another to see if they overlap |
| BoxCollide | Check for a collision with another tlBox |
| CircleCollide | Check for a collision with a tlCircle |
| CircleOverlap | Compare this tlBox with a tlCircle |
| Draw | Draw this tlBox |
| GetCollisionType | Get the collision type of the Box |
| LineCollide | Check for a collision with a tlLine |
| Move | Move the bounding box by a given amount. |
| PointInside | Find out if a point is within the bounding box |
| PolyCollide | Check for a collision with a tlPolygon |
| Position | Set the position of the bounding box. |
| PreventOverlap | Prevent the boundary from overlapping another based on the result of a collision. |
| RayCollide | See is a ray collides with this tlBox |
| ReDimension | Reset the dimensions of the box to a new width and height |
| RectWithin | Find out if a tlBox lies within this objects bounding box |
| Repel | Repel boundaries that overlap |
| Separate | Separate boundaries that overlap |
| SetHandlePosition | Set the handle of the boundary |
| SetScale | Set the scale of the Box |
| SetVelocity | Set the velocity of the boundary |
| SetVelocityVector | Set the velocity vector of the boundary |
| UpdatePosition | Update the position of the boundary |
| UpdateWithinQuadtree | Update the box within the quadatree |
| WorldX | Get the x world coordinate of the boundary |
| WorldY | Get the y world coordinate of the boundary |


Jump here
To test page 2
Hello my name is:
I'm year old!
You have visited this page
Today's Date is
+5
Learning JS part2
when you drop a value in JS array, the gap do not fill. Array in JS is an object. Native to hash table stored as key value, JS wil alwAy return something. undefined. learn arrow function.


Password Generator
Create a safe & secure password below:
Number of Characters
UPPERCASE
lowercase
Numbers
Symbols
Create Password
Copy password
The password has been copied to your clipboard.
Back to v1


Create a table with dynamic col defn : define data type and width as well
Enable Sorting properly for text, date and number cols
Enable pagination
By default the font-size is 16px. Primeng take 1rem unit to size it's icons and other header labels. I feel it is
quite large.
So I reset the the font-size of html to 12px.


Testing Loading Google APIs
Sign in Status:
Unknown
API Key_________:
Client ID_______:
Load GDrive API
Init GDrive Client
Authorize GDrive Access
Remove GDrive Authorization


Provides upcoming air date of any TV show, as well as links to buy and stream past seasons.
@ UGA Intermediate
Here is some more information about this product that is only revealed once clicked on.


The one great certainty of life is death, yet in western society we often have a hard time coming to terms with that. We are dealing with a shifting baseline syndrome, where the essence of death has almost been forgotten. In day to day life, humans are no longer confronted with death in a natural way. There has been a visualization of death in western society that is rooted in fear. There needs to be a systematic shift from disconnect to symbioses that will turn fear into harmony. We need to change the cultural hero system in which human significance is rooted in being alive, and separated from the essence of life itself. Embracing, and re-visualising death as something natural and beautiful could diminish fear, anxiety and most of all reshape our way of living to a more positive, curious and free one. This paper explores manners in which we can achieve exactly that.


It's easy to just pass the callback or return a promise if we don't need to do other stuff to the result.
But what if we wanted to do additional processing? What if we want to read a single line of a file? Then we'll need to add some code to split the file into lines and get the specified line. Let's see how we can do that.
Callbacks
To do this with callbacks, we'll pass our custom callback that does the splitting.
The callback bails-out if there was an error reading the file, otherwise proceeds to split the content and call the original callback with the line content:
function readLine(file, line, callback) {
fs.readFile(file, function process(err, content) {
if (err) return callback(err);
callback(null, content.toString().split('\n')[line]);
});
}
readLine('myfile.txt', 2, function(err, line) {
// handle error or use line
});
Promises
To create a promise-based function we can use another promise method called
.then()
. It works exactly the same as .done()
, but also returns a promise
for the value returned inside its callbacks.
We can simply return the line from inside the first .then
callback. We get a
promise for that line outside of the callback (which we return from the
function)
function readLine(file, line) {
return fs.readFileAsync(file).then(function(res) {
return res.toString().split('\n')[line];
});
}
readLine(file, line).done(function(line) {
// use line
}, function(err) {
// handle error
});
When you call a promise's .then
function, a new promise is created and returned
by .then
. It's a promise to apply all the operations inside the then callback
after the original async operation completes, and return the result.
Notes
In the callback example, we must explicitly handle the error. Since we can't deal with that error there, we must call the passed callback to pass that error.
In the promise example, we can skip the error handling function. If we do that,
the error will automatically propagate with the returned promise for a line.
Yes, that means that you don't have to write if (err) return callback(err)
anymore - promises do the equivalent of that by default.
Another useful way to think of then
vs done
: Promise.done()
is the
equivalent of Array.forEach
while Promise.then()
is the equivalent of
Array.map
. You use the first when you don't want to create a new promise, and
the second when you do.


Running 10K
On running 10K
Today I ran a 10k (10 kilometers, a quarter of a marathon) sponsored by Vienna Running Club in Virginia. I did not prepare actively and only looked up the recommended schedule for it the day before (which convinced me I shouldn’t try a half-marathon that’s doubled the distance). What I did was essentially 1) making sure I was not tired/injured (by not running for the past 3 days), 2) drinking lots of water the day before, and 3) getting a good night sleep. I had been running up for around 5-6 miles at least weekly at around 9 mins/mile pace prior, so my plan was to just keep moving below that pace for the entire duration as much as possible.
Route and pace of 10K from Strava
Generally, I am satisfied with my results. I was with the fast group until around 4 miles when we started going thru hills (which was much more exhausting to me than other runners). I was aware that we sped up after 2 miles (around the turn to Gallows road), but it was easy to keep up as the road was mostly flat. I slowed down a lot around miles 4-5 especially for the hills with steep incline. I plan to work on that along with interval running. I feel tired but happy after the run.
On running with others during COVID-19:
I have been running outdoor every Tuesday with the group since early Summer. I think it is relatively safe compared with indoor sports. Otherwise, I wasn’t very motivated to run (instead I walked with my mom almost every other day).
I think I need the companionship/peer pressure to push myself to exercise more. I don’t socialize much after the run (due partly to COVID-19, my hearing, laziness, and simram), but just having people working towards the same goal is motivating to my mind, especialy for tasks I generally dislike. I have been working on this during meditation.
On Exercise
I will make another post on the exercise routine I’m failing to follow. It is essentially:
- running 3x/week
- bodyweight+core 3x/week (so far I’m only doing this maybe 0.5x/week)
- yoga 1-2x/week


"In geometry, trilateration is the process of determining absolute
or relative locations of points by measurement of distances, using
the geometry of circles, spheres or triangles.
In addition to its interest as a geometric problem, trilateration
does have practical applications in surveying and navigation,
including global positioning systems (GPS). In contrast to
triangulation, it does not involve the measurement of angles."
— Wikipedia
The trilateration can result zero, one or two solutions, in these cases trilaterate() will return null, an Object with { x, y, z } coordinates or an Array with two Objects with { x, y, z } coordinates, respectively.
There is an optional fourth parameter after the three points for trilaterate() for the case of two solutions to return the middle of them as one point.
In this example...
Note: the display of points and distances are projected to the X-Y plane so the Z coordinates cannot be seen, but they are taken into consideration by trilaterate()!


These days, it's pretty common for a single application to use multiple data stores. Sometimes, it's because each data store is specialized for a particular task. For example, an e-commerce site might use Postgres for robust customer data management, Elasticsearch for fast product search, and S3 for cheap image storage. Other times, it's because an app was built by multiple teams over time, so it ends up with one service storing customer data in Postgres and another storing customer data in MongoDB.
Building robust apps with multiple data stores is tricky because it's hard to coordinate operations across those stores. Most databases have a notion of a transaction, a group of related operations which execute as a single atomic, consistent, isolated, and durable unit of work. The canonical example of a transaction is transferring money between bank accounts: to transfer $100 from me to you, the bank runs a transaction which withdraws $100 from my account and deposits $100 in your account. By running both operations in one transaction, we guarantee that either both go through (and the money is transferred) or neither do (and nothing happens), avoiding failure states where $100 disappears from my account but doesn't appear in yours. However, transactions only work within a single database, so if our our system spans multiple data stores, performing operations reliably is much harder. The traditional solution is to use two-phase commit through a protocol like X/Open XA. However, while XA is supported by most big relational databases like Postgres and MySQL, it's not supported by popular newer data stores like MongoDB, Cassandra, or Elasticsearch, even though those stores are increasingly embracing transactions. This means that if you want transactions across multiple data stores, you probably have to do all the hard work of synchronization, concurrency control, and failure management yourself.
In this blog post, I want to tell you about a new protocol named Epoxy (paper), developed as part of my PhD work at Stanford in the DBOS project, which should make this problem easier by providing ACID transactions across heterogeneous data stores. The basic idea behind Epoxy is that we can use a single transactional "primary database", like Postgres, to coordinate transactions among itself and multiple potentially non-transactional "secondary stores", like MongoDB or Elasticsearch. Here's the high-level architecture:
Epoxy works by adapting multi-version concurrency control (MVCC) to a cross-data store setting. We start an Epoxy transaction by initiating a transaction on the primary database. We then ask the primary database for a snapshot, a list of all past primary database transactions whose effects are visible to our new transaction. Typically, that's all transactions that committed before our new transaction began. Here's what a snapshot might look like:
Here, the blue transactions form the snapshot of our new transaction T11, while the pink transactions are invisible to it because they weren't complete when T11 started.
After beginning a transaction, Epoxy interposes on all its reads and writes to secondary stores to enforce snapshot isolation. This provides the abstraction that the transaction operates on a fixed view of data containing only the changes committed by transactions in its snapshot on any data store. Whenever a record is updated on any secondary store, Epoxy doesn't change it in place, but instead creates a new version of it. It tags that version with two pieces of metadata: the ID of the transaction that created the version (beginTxn) and the ID of the transaction that superseded it with a new version (endTxn; this is initialized to infinity, then set when the version is superseded). Epoxy stores this metadata inside the record itself, for example as an additional field in a MongoDB or Elasticsearch document. Then, whenever a read occurs, Epoxy interposes on it, filtering its data source to only include records whose beginTxn field is in the transaction snapshot and whose endTxn field is not. In other words, each transaction can only see the one unique version of a record that was created by a transaction in its snapshot, but not superseded by any transaction in its snapshot. To make that more clear, let's extend our example from earlier and imagine that T11 is a transaction between Postgres and MongoDB:
Here, the blue record versions are visible because they were created but not superseded by transactions in T11's snapshot, while the pink versions are not visible because they were either created by a transaction not in T11's snapshot (like T9) or superseded by a transaction in T11's snapshot (like T8). Thus, Epoxy guarantees that if the effects of a transaction are visible to us in Postgres, they're also visible in MongoDB, and vice versa.
Epoxy also guarantees that transactions are atomic and durable: they either commit on all stores or abort and are rolled back on all stores. After a transaction is done with all operations on all stores, Epoxy validates it, verifying that none of its operations conflict with concurrent committed transactions on any data store (this is a form of optimistic concurrency control). It then asks all secondary stores to persist the transaction's changes to durable storage. If these steps succeed, Epoxy commits the transaction by committing on the primary database. This causes the transaction to appear in future snapshots, atomically making it visible to all future transactions on all data stores. If anything goes wrong, Epoxy aborts, rolling back the transaction on the primary database and undoing its changes on all secondary stores. Even if that process takes a long time, that's fine—none of its changes are visible to any other transactions because, unless committed, they aren't in anyone's snapshots.
Of course, like any new protocol, Epoxy has limitations. First, it makes assumptions about secondary stores, specifically that they provide a way to tag records with metadata and efficiently filter records based on that metadata. Luckily, these assumptions are met by most popular data stores; for example, in MongoDB, you can store all Epoxy metadata in additional document fields and create an index on the field to quickly filter it. Second, Epoxy's interposition comes with some overhead. We analyzed it in our paper—on simulated microservices, Epoxy adds overhead of <10% on read-mostly workloads and ~70% on write-heavy workloads compared to a coordination-free baseline. We found this is similar to the overhead added by XA, even though Epoxy supports more data stores and provides stronger guarantees. Third, and most importantly, Epoxy must be the exclusive mode of accessing a secondary store table because it physically modifies records on secondary stores by adding versioning metadata to them. If one application accessing a secondary store table uses Epoxy, all other applications using that store must use Epoxy for operations on that table. This might make Epoxy tricky to adopt for a database used by multiple services, and it's something that can hopefully be improved in future research.
If you're interested in learning more about Epoxy, please check out our paper (appearing at VLDB 2023). The paper contains a formal description of the protocol, proofs of correctness, and detailed experiments. There's also source code for our research prototype available on GitHub, which implements Epoxy for one primary database (Postgres) and four secondary stores (MongoDB, Elasticsearch, MySQL, and Google Cloud Storage). Let us know what you think!


Classes for testing http server systems. Each test session should begin by creating a {@link com.meterware.httpunit.WebConversation WebConversation} to which it should submit an initial {@link com.meterware.httpunit.GetMethodWebRequest http request} using the {@link com.meterware.httpunit.WebConversation#getResponse getResponse} method. With each subsequent step, it will typically examine the response either textually or as a DOM, and create new requests based on either submitting a form or clicking on a link.
Installation
The package depends on a number of external jar files, provided in the
jar
directory:
-
nekohtml.jar
-
The
NekoHTML parser
, used to convert raw HTML into an XML DOM. This is required for handling HTML.
-
js.jar
-
The
Rhino JavaScript interpreter
, required for any JavaScript processing.
-
xmlParserAPIs.jar
-
The interfaces for a W3-compliant XML parser. Required for interpreting either HTML or XML pages.
-
xercesImpl.jar
-
The
Xerces 2 implementation
of an XML parser. NekoHTML requires this implementation.
-
servlet.jar
-
The APIs and common classes for the Java Servlet 1.3 standard. Required for use with
ServletUnit.
-
junit.jar
-
JUnit
, the unit test framework. Used to test HttpUnit and recommended for writing tests that use HttpUnit.
-
tidy.jar
-
JTidy
, an alternate HTML parser/validator. JTidy is a lot pickier about HTML structure than NekoHTML, and uses its own implementation of the DOM classes, rather than using those found in the xerces jar. Some JavaScript features, such as
document.write()
will only work with NekoHTML.
Example
In the following code, a web conversation is started and an initial request sent. The program then prints out the response and extracts the first form (the login form) from it. After setting the
name
parameter to the desired value, it submits the form and prints the response.
import com.meterware.httpunit.*;
import java.io.IOException;
import java.net.MalformedURLException;
import org.xml.sax.*;
public class Example {
public static void main( String[] params ) {
try {
WebConversation conversation = new WebConversation();
WebResponse response = conversation.getResponse( "http://www.meterware.com/servlet/TopSecret" );
System.out.println( response );
WebForm loginForm = response.getForms()[0];
loginForm.setParameter( "name", "master" );
response = loginForm.submit();
System.out.println( response );
} catch (Exception e) {
System.err.println( "Exception: " + e );
}
}
}
Please direct any questions to
Russell Gold
.


Uncertainty over networks behind delay on French 5G spectrum -FFT
by ReutersPARIS — Uncertainty over which telecom networks could be used is one of the reasons as to why France is likely to delay its 5G spectrum auction, the head of the French telecoms federation said on Wednesday.
Earlier this week, Reuters reported that France is likely to delay its 5G spectrum auction until at least March 2020, three months later than originally scheduled.
“The telecom operators are impatient, but we need clarification on which provider will be used, in order to go forward with our colossal investments in the long term,” Arthur Dreyfuss, head of the FFT French telecoms federation, told BFM Business.
Dreyfuss also said that any plans to ban China’s Huawei from the future French 5G network could lead to even more delays.
“If tomorrow, a network provider was banned, France would have another delay,” he said.
France’s last spectrum auction in 2015 raised 2.8 billion euros ($3.10 billion) for state coffers. Under the current plan, the 5G frequency blocs up for grabs are within the 3.4-3.8 gigahertz bandwidth.
($1 = 0.9033 euros) (Reporting by Sudip Kar-Gupta; Editing by Dominique Vidalon)


Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy. Beer.
Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer. Beer. Beardy Beer.


Trees around the United States.
California.........
New York...........
Texas..............
Not only do trees provide much needed shade on a hot sunny day, they also filter our air and provide stability to the earth.
California.........
New York...........
Texas..............
| =================>TREES<================ | ||||
| TREES | CALIFORNIA | NEW YORK | TEXAS | |
|---|---|---|---|---|
| REDWOODS | Pine | Anacahuita | ||
| California Sycamore | Spruce | Acacia, Sweet |


{# Main template for the application. Args: logged_in whether or not the user is logged in #} {% extends "layout.html" %} {% block title_content %}


Group or Cluster Detection Algorithms¶
This topic has become more and more important for the study of galaxy-halo connection.
Based on clustering of galaxies¶
sfof - Friends-of-Friends Galaxy Cluster Detection Algorithm
By Samuel Farrens. SFoF is a friends-of-friends galaxy cluster detection algorithm that operates in either spectroscopic or photometric redshift space. The linking parameters, both transverse and along the line-of-sight, change as a function of redshift to account for selection effects.
Using the velocity (redshift information)¶
[FOG - identifies locations of clusters by looking for the Finger-of-God effect]
Also in Abdullah et al. 2018 (AWK18) introduced a new technique (GalWeight) to assign cluster membership.
A catalog of 1870 clusters at 0.01 < z < 0.2 from SDSS DR13 (GalWCat19) can be found here
Based on red-sequence of the cluster members¶
redMaPPer: red-sequence Matched-filter Probabilistic Percolation¶
redMaPPer - The Red-sequence Cluster Finder
By Eli Rykoff. This is the open-source, python version of the red-sequence matched-filter Probabilistic Percolation (redMaPPer) cluster finder, originally described in Rykoff et al. (2014), with updates described in Rozo et al. (2015) and Rykoff et al. (2016).
One of the most commonly used algorithm in recent years.


{% extends "base.html" %} {% load static %} {% block title %} Results {% endblock %} {% block header %}
The score is the known as the r-squared value, a percentage representing how well the data was predicted. This value was averaged for the number of folds specified to make sure the results are more reliable
Do you have similarly structured data that you want to predict results for?
Make Predictions


Top 10 sports 1. Football 2. Cricket 3. Basketball 4. Field Hockey 5. Tennis 6. Volleyball 7. Table Tennis 8. Baseball 9. Golf 10. NFL


Welcome to Music Central! Please signup!
Sorry, that username was taken. Please try again. [[${param.param}]]
Choose Username
Password
First Name
Submit


To create a section that goes across the entire width of the screen, use the "section" class on a div.
To create two or more sections next to each other, use the "flex-col" class and the "section" class on a div. Be sure to wrap this in a div containing the "flex-box" class too.
This section is likely for the input of the demo. Users will select options, code, or give values to change the result section of the demo.
Here is where the result of the demo goes. This is to show off what the user has "create" by providing values in the input section of the demo. For example, the input may be a piece of code, and this is the iframe that displays what the code executes as.


Key Word(s): model selection, cross validation, ridge, lasso, regularization
CS109A Introduction to Data Science
Standard Section 4: Regularization and Model Selection¶
Harvard University
Fall 2019
Instructors: Pavlos Protopapas, Kevin Rader, and Chris Tanner
Section Leaders: Marios Mattheakis, Abhimanyu (Abhi) Vasishth, Robbert (Rob) Struyven
#RUN THIS CELL
import requests
from IPython.core.display import HTML
styles = requests.get("http://raw.githubusercontent.com/Harvard-IACS/2018-CS109A/master/content/styles/cs109.css").text
HTML(styles)
For this section, our goal is to get you familiarized with Regularization in Multiple Linear Regression and to start thinking about Model and Hyper-Parameter Selection.
Specifically, we will:
- Load in the King County House Price Dataset
- Perform some basic EDA
- Split the data up into a training, validation, and test set (we'll see why we need a validation set)
- Scale the variables (by standardizing them) and seeing why we need to do this
- Make our multiple & polynomial regression models (like we did in the previous section)
- Learn what regularization is and how it can help
- Understand ridge and lasso regression
- Get an introduction to cross-validation using RidgeCV and LassoCV
# Data and Stats packages
import numpy as np
import pandas as pd
pd.set_option('max_columns', 200)
# Visualization packages
import matplotlib.pyplot as plt
import seaborn as sns
sns.set()
import warnings
warnings.filterwarnings("ignore")
EDA: House Prices Data From Kaggle¶
For our dataset, we'll be using the house price dataset from King County, WA. The dataset is from Kaggle.
The task is to build a regression model to predict the price, based on different attributes. First, let's do some EDA.
# Load the dataset
house_df = pd.read_csv('../data/kc_house_data.csv')
house_df = house_df.sample(frac=1, random_state=42)[0:4000]
print(house_df.shape)
print(house_df.dtypes)
house_df.head()
Now let's check for null values and look at the datatypes within the dataset.
house_df.info()
house_df.describe()
Let's choose a subset of columns here. NOTE: The way I'm selecting columns here is not principled and is just for convenience. In your homework assignments (and in real life), we expect you to choose columns more rigorously.
bedrooms
bathrooms
sqft_living
sqft_lot
floors
sqft_above
sqft_basement
lat
long
price
: Our response variable
cols_of_interest = ['bedrooms', 'bathrooms', 'sqft_living', 'sqft_lot', 'floors', 'sqft_above', 'sqft_basement',
'lat', 'long', 'price']
house_df = house_df[cols_of_interest]
# Convert house price to 1000s of dollars
house_df['price'] = house_df['price']/1000
Let's see how the response variable (price
) is distributed
fig, ax = plt.subplots(figsize=(12,5))
ax.hist(house_df['price'], bins=100)
ax.set_title('Histogram of house price (in 1000s of dollars)');
# This takes a bit of time but is worth it!!
# sns.pairplot(house_df);
Train-Validation-Test Split¶
Up until this point, we have only had a train-test split. Why are we introducing a validation set? What's the point?
This is the general idea:
- Training Set: Data you have seen. You train different types of models with various different hyper-parameters and regularization parameters on this data.
- Validation Set: Used to compare different models. We use this step to tune our hyper-parameters i.e. find the optimal set of hyper-parameters (such as $k$ for k-NN or our $\beta_i$ values or number of degrees of our polynomial for linear regression). Pick your best model here.
- Test Set: Using the best model from the previous step, simply report the score e.g. R^2 score, MSE or any metric that you care about, of that model on your test set.  Why, I hear you ask? Because we want to know how our model might do on data it hasn't seen before. We don't have access to this data (because it may not exist yet) but the test set, which we haven't seen or touched so far, is a good way to mimic this new data.
Let's do 60% train, 20% validation, 20% test for this dataset.
from sklearn.model_selection import train_test_split
# first split the data into a train-test split and don't touch the test set yet
train_df, test_df = train_test_split(house_df, test_size=0.2, random_state=42)
# next, split the training set into a train-validation split
# the test-size is 0.25 since we are splitting 80% of the data into 20% and 60% overall
train_df, val_df = train_test_split(train_df, test_size=0.25, random_state=42)
print('Train Set: {0:0.2f}%'.format(100*train_df.size/house_df.size))
print('Validation Set: {0:0.2f}%'.format(100*val_df.size/house_df.size))
print('Test Set: {0:0.2f}%'.format(100*test_df.size/house_df.size))
Modeling¶
In the last section, we went over the mechanics of Multiple Linear Regression and created models that had interaction terms and polynomial terms. Specifically, we dealt with the following sorts of models.
$$ y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \dots + \beta_M x_M $$
Let's adopt a similar process here and get a few different models.
Creating a Design Matrix¶
From our model setup in the equation in the previous section, we obtain the following:
$$ Y = \begin{bmatrix} y_1 \\ y_2 \\ \vdots \\ y_n \end{bmatrix}, \quad X = \begin{bmatrix} x_{1,1} & x_{1,2} & \dots & x_{1,M} \\ x_{2,1} & x_{2,2} & \dots & x_{2,M} \\ \vdots & \vdots & \ddots & \vdots \\ x_{n,1} & x_{n,2} & \dots & x_{n,M} \\ \end{bmatrix}, \quad \beta = \begin{bmatrix} \beta_1 \\ \beta_2 \\ \vdots \\ \beta_M \end{bmatrix}, \quad \epsilon = \begin{bmatrix} \epsilon_1 \\ \epsilon_2 \\ \vdots \\ \epsilon_n \end{bmatrix}, $$
$X$ is an n$\times$M matrix: this is our design matrix, $\beta$ is an M-dimensional vector (an M$\times$1 matrix), and $Y$ is an n-dimensional vector (an n$\times$1 matrix). In addition, we know that $\epsilon$ is an n-dimensional vector (an n$\times$1 matrix).
X = train_df[cols_of_interest]
y = train_df['price']
print(X.shape)
print(y.shape)
Scaling our Design Matrix¶
Warm-Up Exercise¶
Warm-Up Exercise: for which of the following do the units of the predictors matter (e.g., trip length in minutes vs seconds; temperature in F or C)? A similar question would be: for which of these models do the magnitudes of values taken by different predictors matter?
(We will go over Ridge and Lasso Regression in greater detail later)
- k-NN (Nearest Neighbors regression)
- Linear regression
- Lasso regression
- Ridge regression
Solutions
- kNN: yes. Scaling affects distance metric, which determines what "neighbor" means
- Linear regression: no. Multiply predictor by $c$ -> divide coef by $c$.
- Lasso: yes: If we divided coef by $c$, then corresponding penalty term is also divided by $c$.
- Ridge: yes: Same as Lasso, except penalty divided by $c^2$.
Standard Scaler (Standardization)¶
Here's the scikit-learn implementation of the standard scaler. What is it doing though? Hint: you may have seen this in STAT 110 or another statistics course multiple times.
$$ z = \frac{x-\mu}{\sigma} $$
In the above setup:
- $z$ is the standardized variable
- $x$ is the variable before standardization
- $\mu$ is the mean of the variable before standardization
- $\sigma$ is the standard deviation of the variable before standardization
Let's see an example of how this works:
from sklearn.preprocessing import StandardScaler
x = house_df['sqft_living']
mu = x.mean()
sigma = x.std()
z = (x-mu)/sigma
# reshaping x to be a n by 1 matrix since that's how scikit learn likes data for standardization
x_reshaped = np.array(x).reshape(-1,1)
z_sklearn = StandardScaler().fit_transform(x_reshaped)
# Plotting the histogram of the variable before standardization
fig, ax = plt.subplots(nrows=1, ncols=3, figsize=(24,5))
ax = ax.ravel()
ax[0].hist(x, bins=100)
ax[0].set_title('Histogram of sqft_living before standardization')
ax[1].hist(z, bins=100)
ax[1].set_title('Manually standardizing sqft_living')
ax[2].hist(z_sklearn, bins=100)
ax[2].set_title('Standardizing sqft_living using scikit learn');
# making things a dataframe to check if they work
pd.DataFrame({'x': x, 'z_manual': z, 'z_sklearn': z_sklearn.flatten()}).describe()
Min-Max Scaler (Normalization)¶
Here's the scikit-learn implementation of the standard scaler. What is it doing though?
$$ x_{new} = \frac{x-x_{min}}{x_{max}-x_{min}} $$
In the above setup:
- $x_{new}$ is the normalized variable
- $x$ is the variable before normalized
- $x_{max}$ is the max value of the variable before normalization
- $x_{min}$ is the min value of the variable before normalization
Let's see an example of how this works:
from sklearn.preprocessing import MinMaxScaler
x = house_df['sqft_living']
x_new = (x-x.min())/(x.max()-x.min())
# reshaping x to be a n by 1 matrix since that's how scikit learn likes data for normalization
x_reshaped = np.array(x).reshape(-1,1)
x_new_sklearn = MinMaxScaler().fit_transform(x_reshaped)
# Plotting the histogram of the variable before normalization
fig, ax = plt.subplots(nrows=1, ncols=3, figsize=(24,5))
ax = ax.ravel()
ax[0].hist(x, bins=100)
ax[0].set_title('Histogram of sqft_living before normalization')
ax[1].hist(x_new, bins=100)
ax[1].set_title('Manually normalizing sqft_living')
ax[2].hist(x_new_sklearn, bins=100)
ax[2].set_title('Normalizing sqft_living using scikit learn');
# making things a dataframe to check if they work
pd.DataFrame({'x': x, 'x_new_manual': x_new, 'x_new_sklearn': x_new_sklearn.flatten()}).describe()
The million dollar question
Should I standardize or normalize my data? This, this and this are useful resources that I highly recommend. But in a nutshell, what they say is the following:
Pros of Normalization
- Normalization (which makes your data go from 0-1) is widely used in image processing and computer vision, where pixel intensities are non-negative and are typically scaled from a 0-255 scale to a 0-1 range for a lot of different algorithms.
- Normalization is also very useful in neural networks (which we will see later in the course) as it leads to the algorithms converging faster.
- Normalization is useful when your data does not have a discernible distribution and you are not making assumptions about your data's distribution.
Pros of Standardization
- Standardization maintains outliers (do you see why?) whereas normalization makes outliers less obvious. In applications where outliers are useful, standardization should be done.
- Standardization is useful when you assume your data comes from a Gaussian distribution (or something that is approximately Gaussian).
Some General Advice
- We learn parameters for standardization ($\mu$ and $\sigma$) and for normalization ($x_{min}$ and $x_{max}$). Make sure these parameters are learned on the training set i.e use the training set parameters even when normalizing/standardizing the test set. In sklearn terms, fit your scaler on the training set and use the scaler to transform your test set and validation set (don't re-fit your scaler on test set data!).
- The point of standardization and normalization is to make your variables take on a more manageable scale. You should ideally standardize or normalize all your variables at the same time.
- Standardization and normalization is not always needed and is not an automatic thing you have to do on any data science homework!! Do so sparingly and try to justify why this is needed.
Interpreting Coefficients
A great quote from here
[Standardization] makes it so the intercept term is interpreted as the expected value of 𝑌𝑖 when the predictor values are set to their means. Otherwise, the intercept is interpreted as the expected value of 𝑌𝑖 when the predictors are set to 0, which may not be a realistic or interpretable situation (e.g. what if the predictors were height and weight?)
Standardizing our Design Matrix¶
features = ['bedrooms', 'bathrooms', 'sqft_living', 'sqft_lot', 'floors', 'sqft_above', 'sqft_basement',
'lat', 'long']
X_train = train_df[features]
y_train = np.array(train_df['price']).reshape(-1,1)
X_val = val_df[features]
y_val = np.array(val_df['price']).reshape(-1,1)
X_test = test_df[features]
y_test = np.array(test_df['price']).reshape(-1,1)
scaler = StandardScaler().fit(X_train)
# This converts our matrices into numpy matrices
X_train_t = scaler.transform(X_train)
X_val_t = scaler.transform(X_val)
X_test_t = scaler.transform(X_test)
# Making the numpy matrices pandas dataframes
X_train_df = pd.DataFrame(X_train_t, columns=features)
X_val_df = pd.DataFrame(X_val_t, columns=features)
X_test_df = pd.DataFrame(X_test_t, columns=features)
display(X_train_df.describe())
display(X_val_df.describe())
display(X_test_df.describe())
scaler = StandardScaler().fit(y_test)
y_train = scaler.transform(y_train)
y_val = scaler.transform(y_val)
y_test = scaler.transform(y_test)
One-Degree Polynomial Model¶
import statsmodels.api as sm
from statsmodels.regression.linear_model import OLS
model_1 = OLS(np.array(y_train).reshape(-1,1), sm.add_constant(X_train_df)).fit()
model_1.summary()
Two-Degree Polynomial Model¶
def add_square_terms(df):
df = df.copy()
cols = df.columns.copy()
for col in cols:
df['{}^2'.format(col)] = df[col]**2
return df
X_train_df_2 = add_square_terms(X_train)
X_val_df_2 = add_square_terms(X_val)
# Standardizing our added coefficients
cols = X_train_df_2.columns
scaler = StandardScaler().fit(X_train_df_2)
X_train_df_2 = pd.DataFrame(scaler.transform(X_train_df_2), columns=cols)
X_val_df_2 = pd.DataFrame(scaler.transform(X_val_df_2), columns=cols)
print(X_train_df.shape, X_train_df_2.shape)
# Also check using the describe() function that the mean and standard deviations are the way we want them
X_train_df_2.head()
model_2 = OLS(np.array(y_train).reshape(-1,1), sm.add_constant(X_train_df_2)).fit()
model_2.summary()
Three-Degree Polynomial Model¶
# generalizing our function from above
def add_square_and_cube_terms(df):
df = df.copy()
cols = df.columns.copy()
for col in cols:
df['{}^2'.format(col)] = df[col]**2
df['{}^3'.format(col)] = df[col]**3
return df
X_train_df_3 = add_square_and_cube_terms(X_train_df)
X_val_df_3 = add_square_and_cube_terms(X_val_df)
# Standardizing our added coefficients
cols = X_train_df_3.columns
scaler = StandardScaler().fit(X_train_df_3)
X_train_df_3 = pd.DataFrame(scaler.transform(X_train_df_3), columns=cols)
X_val_df_3 = pd.DataFrame(scaler.transform(X_val_df_3), columns=cols)
print(X_train_df.shape, X_train_df_3.shape)
# Also check using the describe() function that the mean and standard deviations are the way we want them
X_train_df_3.head()
model_3 = OLS(np.array(y_train).reshape(-1,1), sm.add_constant(X_train_df_3)).fit()
model_3.summary()
N-Degree Polynomial Model¶
# generalizing our function from above
def add_higher_order_polynomial_terms(df, N=7):
df = df.copy()
cols = df.columns.copy()
for col in cols:
for i in range(2, N+1):
df['{}^{}'.format(col, i)] = df[col]**i
return df
N = 6
X_train_df_N = add_higher_order_polynomial_terms(X_train_df,N)
X_val_df_N = add_higher_order_polynomial_terms(X_val_df,N)
# Standardizing our added coefficients
cols = X_train_df_N.columns
scaler = StandardScaler().fit(X_train_df_N)
X_train_df_N = pd.DataFrame(scaler.transform(X_train_df_N), columns=cols)
X_val_df_N = pd.DataFrame(scaler.transform(X_val_df_N), columns=cols)
print(X_train_df.shape, X_train_df_N.shape)
# Also check using the describe() function that the mean and standard deviations are the way we want them
X_train_df_N.head()
model_N = OLS(np.array(y_train).reshape(-1,1), sm.add_constant(X_train_df_N)).fit()
model_N.summary()
You can also create a model with interaction terms or any other higher order polynomial term of your choice. Note: Can you see how creating a function that takes in a dataframe and a degree and creates polynomial terms up until that degree can be useful? This is what we have you do in your homework!
from sklearn.metrics import r2_score
x = [1,2,3,N]
models = [model_1, model_2, model_3, model_N]
X_trains = [X_train_df, X_train_df_2, X_train_df_3, X_train_df_N]
X_vals = [X_val_df, X_val_df_2, X_val_df_3, X_val_df_N]
r2_train = []
r2_val = []
for i,model in enumerate(models):
y_pred_tra = model.predict(sm.add_constant(X_trains[i]))
y_pred_val = model.predict(sm.add_constant(X_vals[i]))
r2_train.append(r2_score(y_train, y_pred_tra))
r2_val.append(r2_score(y_val, y_pred_val))
fig, ax = plt.subplots(figsize=(8,6))
ax.plot(x, r2_train, 'o-', label=r'Training $R^2$')
ax.plot(x, r2_val, 'o-', label=r'Validation $R^2$')
ax.set_xlabel('Number of degree of polynomial')
ax.set_ylabel(r'$R^2$ score')
ax.set_title(r'$R^2$ score vs polynomial degree')
ax.legend();
We notice a big difference between training and validation R^2 scores: seems like we are overfitting. Introducing: regularization.
What about Multicollinearity?¶
There's seemingly a lot of multicollinearity in the data. Take a look at this warning that we got when showing our summary for our polynomial models:
What is multicollinearity? Why do we have it in our dataset? Why is this a problem?
Does regularization help solve the issue of multicollinearity?
What does Regularization help with?¶
We have some pretty large and extreme coefficient values in our most recent models. These coefficient values also have very high variance. We can also clearly see some overfitting to the training set. In order to reduce the coefficients of our parameters, we can introduce a penalty term that penalizes some of these extreme coefficient values. Specifically, regularization helps us:
- Avoid overfitting. Reduce features that have weak predictive power.
- Discourage the use of a model that is too complex
Big Idea: Reduce Variance by Increasing Bias¶
Image Source: here
Ridge Regression¶
Ridge Regression is one such form of regularization. In practice, the ridge estimator reduces the complexity of the model by shrinking the coefficients, but it doesn’t nullify them. We control the amount of regularization using a parameter $\lambda$. NOTE: sklearn's ridge regression package represents this $\lambda$ using a parameter alpha. In Ridge Regression, the penalty term is proportional to the L2-norm of the coefficients.
Lasso Regression¶
Lasso Regression is another form of regularization. Again, we control the amount of regularization using a parameter $\lambda$. NOTE: sklearn's lasso regression package represents this $\lambda$ using a parameter alpha. In Lasso Regression, the penalty term is proportional to the L1-norm of the coefficients.
Some Differences between Ridge and Lasso Regression¶
- Since Lasso regression tend to produce zero estimates for a number of model parameters - we say that Lasso solutions are sparse - we consider to be a method for variable selection.
- In Ridge Regression, the penalty term is proportional to the L2-norm of the coefficients whereas in Lasso Regression, the penalty term is proportional to the L1-norm of the coefficients.
- Ridge Regression has a closed form solution! Lasso Regression does not. We often have to solve this iteratively. In the sklearn package for Lasso regression, there is a parameter called
max_iter
that determines how many iterations we perform.
Why Standardizing Variables was not a waste of time¶
Lasso regression puts constraints on the size of the coefficients associated to each variable. However, this value will depend on the magnitude of each variable. It is therefore necessary to standardize the variables.
Let's use Ridge and Lasso to regularize our degree 10 polynomial¶
Exercise: Play around with different values of alpha. Notice the new $R^2$ value and also the range of values that the predictors take in the plot.
from sklearn.linear_model import Ridge
# some values you can try out: 0.01, 0.1, 0.5, 1, 5, 10, 20, 40, 100, 200, 500, 1000, 10000
alpha = 100
ridge_model = Ridge(alpha=alpha).fit(X_train_df_N, y_train)
print('R squared score for our original OLS model: {}'.format(r2_val[-1]))
print('R squared score for Ridge with alpha={}: {}'.format(alpha, ridge_model.score(X_val_df_N,y_val)))
fig, ax = plt.subplots(figsize=(18,8), ncols=2)
ax = ax.ravel()
ax[0].hist(model_N.params, bins=10, alpha=0.5)
ax[0].set_title('Histogram of predictor values for Original model with N: {}'.format(N))
ax[0].set_xlabel('Predictor values')
ax[0].set_ylabel('Frequency')
ax[1].hist(ridge_model.coef_.flatten(), bins=20, alpha=0.5)
ax[1].set_title('Histogram of predictor values for Ridge Model with alpha: {}'.format(alpha))
ax[1].set_xlabel('Predictor values')
ax[1].set_ylabel('Frequency');
from sklearn.linear_model import Lasso
# some values you can try out: 0.00001, 0.0001, 0.001, 0.01, 0.05, 0.1, 0.5, 1, 2, 5, 10, 20
alpha = 0.01
lasso_model = Lasso(alpha=alpha, max_iter = 1000).fit(X_train_df_N, y_train)
print('R squared score for our original OLS model: {}'.format(r2_val[-1]))
print('R squared score for Lasso with alpha={}: {}'.format(alpha, lasso_model.score(X_val_df_N,y_val)))
fig, ax = plt.subplots(figsize=(18,8), ncols=2)
ax = ax.ravel()
ax[0].hist(model_N.params, bins=10, alpha=0.5)
ax[0].set_title('Histogram of predictor values for Original model with N: {}'.format(N))
ax[0].set_xlabel('Predictor values')
ax[0].set_ylabel('Frequency')
ax[1].hist(lasso_model.coef_.flatten(), bins=20, alpha=0.5)
ax[1].set_title('Histogram of predictor values for Lasso Model with alpha: {}'.format(alpha))
ax[1].set_xlabel('Predictor values')
ax[1].set_ylabel('Frequency');
Model Selection and Cross-Validation¶
Here's our current setup so far:
So we try out 10,000 different models on our validation set and pick the one that's the best? No! Since we could also be overfitting the validation set!
One solution to the problems raised by using a single validation set is to evaluate each model on multiple validation sets and average the validation performance. This is the essence of cross-validation!
Image source: here
from sklearn.linear_model import RidgeCV
from sklearn.linear_model import LassoCV
alphas = (0.001, 0.01, 0.1, 10, 100, 1000, 10000)
# Let us do k-fold cross validation
k = 4
fitted_ridge = RidgeCV(alphas=alphas).fit(X_train_df_N, y_train)
fitted_lasso = LassoCV(alphas=alphas).fit(X_train_df_N, y_train)
print('R^2 score for our original OLS model: {}\n'.format(r2_val[-1]))
ridge_a = fitted_ridge.alpha_
print('Best alpha for ridge: {}'.format(ridge_a))
print('R^2 score for Ridge with alpha={}: {}\n'.format(ridge_a, fitted_ridge.score(X_val_df_N,y_val)))
lasso_a = fitted_lasso.alpha_
print('Best alpha for lasso: {}'.format(lasso_a))
print('R squared score for Lasso with alpha={}: {}'.format(lasso_a, fitted_lasso.score(X_val_df_N,y_val)))
We can also look at the coefficients of our CV models.
Final Step: report the score on the test set for the model you have chosen to be the best.


Research and develop software and hardware. Program embedded system and develop electronic system. Target fields include desktop applications, web applications and Android apps.
Technical Support
Diagnose hardware and software faults. Solve technical and applications problems.
Data Analytics and AI
Looking to offer services in the area of Data Science, Deep Learning and Predictive Analytics for Business.
Portfolio
Desktop Application. Electronics Projects. Web Application. Android App.
Exhibit Presentation System (EPS)
Real-time display of evidence and annotation of image during a court case.
Queue Management System
Provide and cater ticketing for queue and counter services.
Wireless Gateway Module for Safety Helmet
Wirelessly monitor worker's condition working in confined-space.
Indoor Child Tracking System
Alert parents when child is out of parent-child proximity.
Automatic Plant Watering System
A smart customizable plant watering system.
Traffic Light System
A smart traffic light system design using Verilog and Altera FPGA DE2 board.
VLSI Design of 4-bit counter
Noise Reduction Audio Processing
Process audio and reduce noise using MATLAB FDA tool.
Design of Digital 7 Tap FIR Filter
A Finite Impulse Response (FIR) filter design using RTL Design Methodology.
Development and Enhancement of Corporate Building Work and Attendance Management System Web Applications and Services in Symfony, Bootstrap, HTML5, CSS3, PHP, Javascript etc
Development of a GUI Bank Website using Python, Tkinter and MySQL
Development of Automatic Battery Cut-off Charger Prototype. Programmed using Assembly Language on Wise Atmel AVR SDK Board
Development of Door Lock System using Altera Quartus II
Development of AutoCar, constructed and programmed using C in MPLAB IDE
Exhibit Presentation System (EPS)
Real-time display of evidence and annotation of image during a court case.
Below is a testing video during software development.
Exhibit Presentation System (EPS) consists of a Controller screen and Editor screen. The interpreter uses the Controller screen to:
1. Select court case, exhibits and retrieve evidence images
2. Capture or open evidence image
3. Create new or clone exhibits
4. Select Editor user, Editor annotation color and drawing ball size
5. Manipulate and extend evidence image to Editor Screen
6. Control over Editor Screen
7. Export annotated evidence image to PDF locally or back to EPS database
The defendant and witnesses use the Editor screen to annotate extended image.
Date: January 2019
Client: The High Court in Sabah and Sarawak
Category: Desktop Application. Windows Application. IPEVO Visualizer. JSON. C#.
Queue Management System (QMS)
Provide and cater ticketing for queue and counter services.
Above is the counter ticketing PUBLIC DISPLAY.
Above is my teammates who are installing COUNTER into the desktops.
Above is the desktops having our COUNTER software installed.
Above is when I was doing debugging.
Above is our QMS been used by one of our clients - Padawan Municipal Council.
Queue Management System (QMS) consist of softwares KIOSK, COUNTER, PUBLIC DISPLAY and RATING.
KIOSK - A standalone machine that allow citizen to select service and get queue ticket.
COUNTER - For government workers to CALL, CANCEL, PASS, WAIT, RECALL and END ticket from citizen.
PUBLIC DISPLAY - To display current and past ticket numbers and counters.
RATING - For citizen to rate the government worker's services after the service ends.
Date: April 2017
Clients: 1. Padawan Municipal Council 2. Kuching Resident and District Office
Category: Desktop Application. Windows Application. C#. MYSQL. Windows Services. Web Services.
Wireless Gateway Module for Safety Helmet
Wirelessly monitor worker's condition working in confined-space.
Above is the project overview.
Above is the software look.
Above is the award and recognition received for this project.
Above is the demostration of the project.
Wireless Gateway Module for Safety Helmet is a wireless monitoring system aim to monitor worker working in
confined space and unreachable environment. The system consists of a software and a worker helmet. The wireless
gateway module live-streams and monitors heart rate, SPO2, relative humidity and temperature. It sends the data to
the software via IEEE 802.11 WiFi.
Alert parents when child is out of parent-child proximity.
Above is the project overview. The video below describes how it works.
Indoor Child Tracking System consists of a Child Tracker Watch and an Android App. When the connection between
the watch and the app is established, the tracking system is activated. When the watch worn by a child exceeds the
Bluetooth range away from the app, both the device and the app is alarmed. The app would notify the parents about
the child lost and the track-back is started.
Date: December 2015
Scope: Undergraduate Studies 3rd Year Capstone Project
Category: Android App. Electronic Project. Bluetooth module HC-05. C Programming Language. MIT App Inventor.
Automatic Plant Watering System
A smart customizable plant watering system.
Above is the project demostration.
FUNCTIONS
1. Water when the soil moisture value is low (~100).
2. User able to adjust the threshold value (TV). System water when TV < Soil Moisture.
3. User able to on/off button to water manually.
Date: May 2016
Scope: Undergraduate Studies 3rd Year Embedded System Project
Category: Electronic Project. Arduino IDE. C Programming Language. Intel Galileo Gen 2 Development Board.
16X2 LCD Display. DC 9V Water Pump Motor with Pipe. Relay 6V. 1N4001 diode. Potentiometer.
Traffic Light System
A smart traffic light system design using Verilog and Altera FPGA DE2 board.
Download the project task here.
Download the project report here.
Referring to the diagram above, the traffic light at a junction between a main road and a small road is controlled
by two motion sensors M1 or M2.
The two main traffic lights, SA and SB take turns to change to green every 12 seconds once.
However, if there is a car detected by motion sensors, M1 or M2, then after the current main traffic light changes
to red, SC or SD are changed to green for 12 seconds followed by the resumed process of change between SA and SB.
If both motion sensors are activated at the same time, SC is given priority to become green for 12 seconds followed
by SD for 12 seconds.
Date: March 2015
Scope: Undergraduate Studies 3rd Year VLSI Project
Category: VLSI. Cadence. Verilog. Altera Quartus II. Altera FPGA DE2 board.
VLSI Design of 4-bit counter
To VLSI design 4-bit counter that counts from 0 to 8 and back to 0.
Download the project report here.
Date: June 2015
Scope: Undergraduate Studies 3rd Year VLSI Project
Category: VLSI. Cadence EDA. Verilog. Altera Quartus II.
Noise Reduction Audio Processing
Process audio and reduce noise using MATLAB FDA tool.
This is the original audio clip.
This is the filtered audio clip.
Download the project task here.
Download the project presentation here.
Date: June 2015
Scope: Undergraduate Studies 3rd Year Digital Signal Processing Project
Category: MATLAB FDA Tool
Design of Digital 7 Tap FIR Filter
A digital Finite Impulse Response (FIR) filter design using RTL Design Methodology.
Digital Finite Impulse Response (FIR) filters are the basic building block of many digital signal processing (DSP)
systems. FIR filters are used due their stability, easy to accomplish linear phase and can be realized efficiently
in hardware. The main objectives of digital FIR filter are to filter out undesirable parts of the signal, shape of
the spectrum of signals in communication channels, signal detection or analysis in radar applications. Adders,
multipliers and delay elements are the main components used in the implementation of digital FIR filters.
Download the project report here.
Date: May 2015
Scope: Undergraduate Studies 3rd Year CAD & HDL Project
Use this area to describe your project. Lorem ipsum dolor sit amet, consectetur adipisicing elit. Est blanditiis dolorem culpa incidunt minus dignissimos deserunt repellat aperiam quasi sunt officia expedita beatae cupiditate, maiores repudiandae, nostrum, reiciendis facere nemo!
Date: January 2017
Client: Threads
Category: Illustration
Project Name
Lorem ipsum dolor sit amet consectetur.
 Lorem ipsum dolor sit amet, consectetur adipisicing elit. Est blanditiis dolorem culpa incidunt minus dignissimos deserunt repellat aperiam quasi sunt officia expedita beatae cupiditate, maiores repudiandae, nostrum, reiciendis facere nemo!
Date: January 2017
Client: Threads
Category: Illustration
Project Name
Lorem ipsum dolor sit amet consectetur.
 Lorem ipsum dolor sit amet, consectetur adipisicing elit. Est blanditiis dolorem culpa incidunt minus dignissimos deserunt repellat aperiam quasi sunt officia expedita beatae cupiditate, maiores repudiandae, nostrum, reiciendis facere nemo!
Date: January 2017
Client: Threads
Category: Illustration
Project Name
Lorem ipsum dolor sit amet consectetur.
 Lorem ipsum dolor sit amet, consectetur adipisicing elit. Est blanditiis dolorem culpa incidunt minus dignissimos deserunt repellat aperiam quasi sunt officia expedita beatae cupiditate, maiores repudiandae, nostrum, reiciendis facere nemo!


| functionId [ ( [ expn { , expn } ] ) ] |
function firstWord ( str : string ): string
for i : 1 .. length ( str )
if str ( i ) = " " then
result str ( 1 .. i - 1)
end if
end for
end firstWord
put "The first word is: ", firstWord ("Henry Hudson")
% The output is Henry.
Each actual parameter passed to its non-var formal parameter must be assignable to that parameter (see assignability for details). See also functionDeclaration and procedureDeclaration.
In this explanation of functionCall, we have up to this point ignored the possibility of functions exported from modules. If the function is being called from outside of a module from which it has been exported, the syntax of the functionCall is:
In other words, the module or monitor name and a dot must precede the function's name. If the function is being called from outside of a class from which it has been exported, the syntax of the functionCall is one of:
In these p must be a pointer value that locates an object in the class. Form (b) is a short form for form (a).
moduleId . functionId [ ( expn {, expn } ) ]
(a) classId ( p ) . functionId [ ( [ expn {, expn } ] ) ]
(b) p -> functionId [ ( [ expn {, expn } ] ) ]


Let's find a match!
Dating is a stage of romantic relationships in humans whereby two people meet socially with the aim of each assessing the other's suitability as a prospective partner in an intimate relationship. It is a form of courtship, consisting of social activities done by the couple, either alone or with others.


class: center, middle Eloress The Digital Witch --- # Definitions: elora; The glow of any electronic device, gadget or light source. Incandescent light bulbs, Candles, and Fluorescent lights do not emit an elora. LED, LCD, CRT, and Plasma light sources all do emit an elora. The elora of the cell phone after the power had gone out was both eery and reassuring. eloress; A sorceress of the electronic. A digital witch. A person with an affinity to technology. ??? I coined the term for myself and it is therefore specific to the female gender. Sorceress implies female sorceror, Likewise Eloress implies female self identity. However, I have no issues with male identified people self identifying as such, or genderqueer either. Genderqueer and male identified people are also welcome to think up an appropriate male or genderqueer suffix or modify the term in any way they see fit. The gender specificity is not relevant to the lesson. --- # Elaboration: Definitions of Shamen, Shameness are somewhat bound by tradition, History, and Cultural appreciation/appropriation. Eloress however is not bound by any definition excepting that which I create for it. And So, Eloress, one who uses technology to weave magic. ??? This can include performing music, and modifying previously recorded music via playback. The pretentious idea that a DJ is a Dancefloor Shamen is both disrespectful and absurd as most DJs have no interest, education, or capacity to heal through music. --- A Eloress can be a DJ, and music performer and in such a capacity is at their most illustrative. In this respect, an Eloress considers, binaural audio, rythem, frequency, and cultural cues. --- An Eloress is also one who seeks to understand and work with and not against their technological tools. An Eloress has the ability to diagnose and fix issues within the tools they use and as a service to their community. She is for herself a tinkerer and for others a "Computer doctor". One of her most important aspects in this is teaching others how to get along with their devices and understand that computers are not faultless, but bound by the same physical parameters we and everything else in the world is.