Kubernetes network policies are a useful security feature which allow for traffic into and (sometimes) out of pods to be restricted.
This is very useful if you want to add another layer of defence to your cluster and reduce the risk of attacks both on other services running on the cluster and also the control plane services like etcd and the Kubelet.
To make use of network policies, you need to have a k8s version that supports them (the Network Policy API hit stable in 1.7). You’ll also need a network plugin that supports Network Policies, in order for your policies to be effective. Most of the major network plugins support network policies, however there are some irregularities to be aware of.
For example Weave doesn’t currently support egress policies at the moment (issue here ), so do check the support of your chosen plugin before starting :)
Network policies are a lot like network ACLs or firewall rules, if you’re familiar with those. Without any network policies in effect on a set of pods, there’s a “default allow” rule in place. However as soon as any network Policy applies to a given pod, that pod has a “default deny” setup applied, meaning you have to specify all the traffic desired for that pod once you’ve started implementing network policies on it.
There are two types of network policies that can be specified. Ingress policies restrict traffic to set of pods, and egress policies restrict outbound traffic from a set of pods.
The Kubernetes documentation on network policies has some good examples of policies you might want to apply, and there’s also a repo. on github with some more examples, with a nice visualization of the effect of the policy, however lets cover one example that’s not covered in either of those resources.
One of the challenges I’ve seen in assessments of Kubernetes cluster security, when we work from a “compromised container” perspective, is that it’s possible to attack the underlying nodes and this exposes the control plane services to attack. It’s not uncommon to see unauthenticated access to etcd or the Kubelet, and unauthorised access to those services is pretty bad for the overall security of the cluster.
So can we use network policies to prevent this? The answer seems to be “yes, but with some side effects”.
I’ve got a demo 3 node 1.9 cluster using Calico for the network plugin. The host network is on <IP_ADDRESS>/24
so the goal of my network policy is to restrict access to that network, whilst still allowing the pods to communicate with the rest of the world.
I’ve got a namespace setup for this test (netpol-test) and a sample alpine container running there, so we can check the results of our network policies.
So at a basic level we can use something like this
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: default-block
namespace: netpol-test
spec:
podSelector: {}
egress:
- to:
- ipBlock:
cidr: <IP_ADDRESS>/0
except:
- <IP_ADDRESS>/24
policyTypes:
- Egress
to restrict outbound access. What you can see here is that we’re applying the network policy to a specific namespace with namespace: netpol-test
and then we’re using a blank pod selector podSelector: {}
to hit all the pods in that namespace.
Then we’re specifying an egress policy. Network policies are all about allowing traffic after the initial “default deny” is in place, so we have to specify our restriction in a kind of back-handed way.
We allow all destinations with cidr: <IP_ADDRESS>/0
and then block a specific network with the except:
block.
If you look at the effects before and after using something like nmap, you’ll see that ports like 10250/TCP
which were accessible before applying the policy, are no longer visible afterwards.
So what about the side effects that I mentioned above? Well when I was testing this network policy, I noticed that in addition to blocking access to the nodes directly (on <IP_ADDRESS>/24) this policy also prevents access to the Kubernetes service, which in the case of this cluster is on <IP_ADDRESS>:443.
This could be desirable from a security standpoint, as it’s blocking access to control plane services, but it is rather un-intuitive to have a completely different network blocked based on a network policy.
So in this model where we’re using network policies to restrict control plane service access, the one service we may want to allow containers to speak to is the Kubernetes API. As the access provided by network policies is cumulative, this is pretty straightforward. Working with the same setup as the previous example, we can just apply something like this
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: allow-api
namespace: netpol-test
spec:
podSelector: {}
egress:
- to:
- ipBlock:
cidr: <IP_ADDRESS>/24
ports:
- protocol: TCP
port: 6443
policyTypes:
- Egress
and we’ll have access to the API service from pods in the netpol-test
namespace. Also our oddness from the previous example is present here as well. Applying this policy also opens up access to <IP_ADDRESS>:443 even though that’s not the port or IP address specified!
This was just a brief introduction to Network policies in Kubernetes, there’s a lot more that can be done with them in terms of allowing access between specific pods and services. Overall they’re a good layer of additional protection, and one that’s well worth considering for production clusters, although I have a feeling that if used heavily, management of the rulesets could get a bit complex…


Emilia Clarke says she was uncomfortable with Game of Thrones nude scenes and was even in tears before some
Emilia Clarke has revealed she felt uncomfortable with some of her nude scenes in Game Of Thrones and was even in tears before some of them.
The British actress played queen Daenerys Targaryen in HBO's sprawling epic, which became notorious for its explicit portrayal of sex and violence.
Clarke, who was 23 when she started filming Game Of Thrones, said she would be in tears before shooting certain nude scenes.
- Game of Thrones star Emilia Clarke reveals she suffered two life-threatening brain aneurysms
- 'House of the Dragon': Game of Thrones prequel series announced
- Game of Thrones star Jason Momoa shaves off his beard - leaving fans distraught
Speaking on actor Dax Shepard's podcast, she said: "I took the job and then they sent me the scripts and I was reading them, and I was like, 'Oh, there's the catch!'
"But I'd come fresh from drama school and I approached it as a job, if it's in the script then it's clearly needed, this is what this is and I'm going to make sense of it. Everything's going to be cool."
Clarke, 33, added: "I've never been on a film set like this before, I'd been on a film set twice before then, and I'm now on a film set completely naked with all of these people, and I don't know what I'm meant to do, and I don't know what's expected of me, and I don't know what you want, and I don't know what I want.
"Regardless of there being nudity or not, I would have spent that first season thinking I'm not worthy of requiring anything; I'm not worthy of needing anything at all."
However Clarke, who credited her Game Of Thrones co-star Jason Momoa with encouraging her to speak up about her reservations, said she became more "savvy" in her later roles.
She said: "I've had fights on set before where I'm like, 'No, the sheet stays up,' and they're like, 'You don't wanna disappoint your Game of Thrones fans.' And I'm like, 'F*** you.'"
Representatives for HBO and Game Of Thrones creators David Benioff and DB Weiss have been contacted for comment.


Flights suspended at Chattogram airport for 14 hours ahead of cyclone
by bdnews24.comThe authorities have suspended all flights to and from Chattogram’s Shah Amanat International Airport for 14 hours from Saturday afternoon as Bangladesh braces for Cyclone Bulbul.
However, flights operations at Barishal, Jashore and Cox's Bazar airports are unaffected, according to officials.
Bangladesh Meteorology Department issued great danger signal No. 10 for the maritime ports of Payra and Mongla while the Chattogram port has been advised to signal No. 9.
The ‘very severe’ cyclonic storm is expected to cross the Sundarbans delta on Saturday evening, according to meteorologists.
Under the circumstances, no flights will operate in and out of Chattogram airport until 6am on Sunday, said AKM Rezaul Kabir, the port-city airport’s spokesperson.
The Sylhet airport -- which usually remains closed at night -- will be kept open to manage the air traffic.
No decision has yet been taken to on the operations at the Jashore, Cox's Bazar and Barishal airports. A decision will be taken after evaluating the local weather situation, an official said.
Barishal airport’s Manager Rathindra Nath Chowdhury said, "Flight operations will continue as usual at our airport. If someone wants to operate a flight, there is no barrier.”
"There is no directive to suspend the flights. From morning until now, six aircraft have taken off,” Jashore airport’s Manager Masudul Haque said.


The following tables list the members that are exposed by the SQLServerDatabaseMetaData class.
Constructors
None.
Fields
None.
Inherited Fields
| Name | Description |
|---|---|
|
java.sql.DatabaseMetaData |
attributeNoNulls, attributeNullable, attributeNullableUnknown, bestRowNotPseudo, bestRowPseudo, bestRowSession, bestRowTemporary, bestRowTransaction, bestRowUnknown, columnNoNulls, columnNullable, columnNullableUnknown, importedKeyCascade, importedKeyInitiallyDeferred, importedKeyInitiallyImmediate, importedKeyNoAction, importedKeyNotDeferrable, importedKeyRestrict, importedKeySetDefault, importedKeySetNull, procedureColumnIn, procedureColumnInOut, procedureColumnOut, procedureColumnResult, procedureColumnReturn, procedureColumnUnknown, procedureNoNulls, procedureNoResult, procedureNullable, procedureNullableUnknown, procedureResultUnknown, procedureReturnsResult, sqlStateSQL, sqlStateSQL99, sqlStateXOpen, tableIndexClustered, tableIndexHashed, tableIndexOther, tableIndexStatistic, typeNoNulls, typeNullable, typeNullableUnknown, typePredBasic, typePredChar, typePredNone, typeSearchable, versionColumnNotPseudo, versionColumnPseudo, versionColumnUnknown |
Methods
| Name | Description |
|---|---|
|
Retrieves whether the current user has permissions to call all the procedures returned by the getProcedures method. |
|
|
Retrieves whether the current user has permissions to use all the tables returned by the getTables method in a SELECT statement. |
|
|
Indicates whether the JDBC driver closes all the open result sets, including the holdable ones, when an auto-commit is enabled and an exception is raised. |
|
|
Retrieves whether a data definition statement within a transaction forces the transaction to commit. |
|
|
Retrieves whether this database ignores a data definition statement within a transaction. |
|
|
Retrieves whether or not a visible row delete can be detected by calling the rowDeleted method of the SQLServerResultSet class. |
|
|
Retrieves whether the return value for the getMaxRowSize method includes the SQL data types LONGVARCHAR and LONGVARBINARY. |
|
|
Retrieves a description of the given attribute of the given type for a user-defined type that is available in the given schema and catalog. |
|
|
Retrieves a description of the optimal set of columns of a table that uniquely identifies a row. |
|
|
Retrieves the catalog names that are available in the connected server. |
|
|
Retrieves the String that this database uses as the separator between a catalog and table name. |
|
|
Retrieves the database vendor's preferred term for "catalog". |
|
|
Retrieves a list of the client information properties that the driver supports. |
|
|
Retrieves a description of the access rights for the columns in a table. |
|
|
Retrieves a description of the table columns that are available in the specified catalog. |
|
|
Retrieves the connection that produced this metadata object. |
|
|
Retrieves a description of the foreign key columns in the given foreign key table that references the primary key columns of the given primary key table. |
|
|
Retrieves the major version number of the underlying database. |
|
|
Retrieves the minor version number of the underlying database. |
|
|
Retrieves the name of this database product. |
|
|
Retrieves the version number of this database product. |
|
|
Retrieves the default transaction isolation level for this database. |
|
|
Retrieves the major version number of this JDBC driver. |
|
|
Retrieves the minor version number of this JDBC driver. |
|
|
Retrieves the name of this JDBC driver. |
|
|
Retrieves the version number of this JDBC driver. |
|
|
Retrieves a description of the foreign key columns that reference the given table's primary key columns. |
|
|
Retrieves all the extra characters that can be used in unquoted identifier names, for example, those beyond a-z, A-Z, 0-9, and _. |
|
|
Retrieves a description of the system and user functions. |
|
|
Retrieves a description of the specified catalog's system- or user-function parameters and return type. |
|
|
Retrieves the String that is used to quote SQL identifiers. |
|
|
Retrieves a description of the primary key columns that are referenced by a table's foreign key columns. |
|
|
Retrieves a description of the indexes and statistics of the given table. |
|
|
Retrieves the major JDBC version number for this driver. |
|
|
Retrieves the minor JDBC version number for this driver. |
|
|
Retrieves the maximum number of hex characters that this database allows in an inline binary literal. |
|
|
Retrieves the maximum number of characters that this database allows in a catalog name. |
|
|
Retrieves the maximum number of characters that this database allows for a character literal. |
|
|
Retrieves the maximum number of characters that this database allows for a column name. |
|
|
Retrieves the maximum number of columns that this database allows in a GROUP BY clause. |
|
|
Retrieves the maximum number of columns that this database allows in an index. |
|
|
Retrieves the maximum number of columns that this database allows in an ORDER BY clause. |
|
|
Retrieves the maximum number of columns that this database allows in a SELECT list. |
|
|
Retrieves the maximum number of columns that this database allows in a table. |
|
|
Retrieves the maximum number of concurrent connections to this database that are possible. |
|
|
Retrieves the maximum number of characters that this database allows in a cursor name. |
|
|
Retrieves the maximum number of bytes that this database allows for an index, including all of the parts of the index. |
|
|
Retrieves the maximum number of characters that this database allows in a procedure name. |
|
|
Retrieves the maximum number of bytes that this database allows in a single row. |
|
|
Retrieves the maximum number of characters that this database allows in a schema name. |
|
|
Retrieves the maximum number of characters that this database allows in an SQL statement. |
|
|
Retrieves the maximum number of active statements to this database that can be open at the same time. |
|
|
Retrieves the maximum number of characters that this database allows in a table name. |
|
|
Retrieves the maximum number of tables that this database allows in a SELECT statement. |
|
|
Retrieves the maximum number of characters that this database allows in a user name. |
|
|
Retrieves a comma-separated list of math functions that are available with this database. |
|
|
Retrieves a description of the primary key columns of the given table. |
|
|
Retrieves a description of the stored procedure parameters and result columns. |
|
|
Retrieves a description of the stored procedures that are available in the given catalog, schema, or stored procedure name pattern. |
|
|
Retrieves the preferred term for "procedure" in this database. |
|
|
Retrieves the default holdability of result sets for this database. |
|
|
Returns a status indicating whether or not SQL RowId data type is supported. If supported, it returns the lifetime for which a RowId object remains valid. |
|
|
Retrieves the schema names that are available in the current database. |
|
|
Retrieves the preferred term for "schema" in this database. |
|
|
Retrieves the String that can be used to escape wildcard characters. |
|
|
Retrieves a comma-separated list of all of this database's SQL keywords that are not also SQL92 keywords. |
|
|
Indicates whether the SQLSTATE returned by the SQLException.getSQLState method is X/Open (now known as Open Group), SQL CLI, SQL99 (JDBC 3.0), or SQL:2003 (JDBC 4.0). |
|
|
Retrieves a comma-separated list of String functions that are available with this database. |
|
|
Retrieves a description of the table hierarchies that are defined in a particular schema in this database. |
|
|
Retrieves a description of the user-defined type hierarchies that are defined in a particular schema in this database. |
|
|
Retrieves a comma-separated list of system functions that are available with this database. |
|
|
Retrieves a description of the access rights for each table that is available in the given catalog, schema, or table name pattern. |
|
|
Retrieves a description of the tables that are available in the given catalog, schema, or table name pattern. |
|
|
Retrieves the table types that are available in the current database. |
|
|
Retrieves a comma-separated list of the time and date functions that are available with this database. |
|
|
Retrieves a description of all the standard SQL types that are supported by the current database. |
|
|
Retrieves a description of the user-defined types that are defined in a particular schema. |
|
|
Retrieves the URL for this database. |
|
|
Retrieves the user name as known to this database. |
|
|
Retrieves a description of the columns of a table that is automatically updated when any value in a row is updated. |
|
|
Retrieves whether or not a visible row insert can be detected by calling the method rowInserted method of the SQLServerResultSet class. |
|
|
Retrieves whether a catalog appears at the start of a fully qualified table name. |
|
|
Retrieves whether this database is in read-only mode. |
|
|
Indicates whether updates made to a LOB are made on a copy or directly to the LOB. |
|
|
Indicates whether this database supports concatenations between NULL and non-NULL values being NULL. |
|
|
Retrieves whether NULL values are sorted at the end regardless of sort order. |
|
|
Retrieves whether NULL values are sorted at the start regardless of sort order. |
|
|
Retrieves whether NULL values are sorted high. |
|
|
Retrieves whether NULL values are sorted low. |
|
|
Retrieves whether deletes that are made by others are visible. |
|
|
Retrieves whether inserts that are made by others are visible. |
|
|
Retrieves whether updates that are made by others are visible. |
|
|
Retrieves whether a result set's own deletes are visible. |
|
|
Retrieves whether a result set's own inserts are visible. |
|
|
Retrieves whether the result set's own updates are visible. |
|
|
Retrieves whether this database treats mixed-case SQL identifiers that are not enclosed in quotation marks as case-insensitive and stores them in lowercase. |
|
|
Retrieves whether this database treats mixed-case SQL identifiers that are enclosed in quotation marks as case-insensitive and stores them in lowercase. |
|
|
Retrieves whether this database treats mixed-case SQL identifiers that are not enclosed in quotation marks as case-insensitive and stores them in mixed case. |
|
|
Retrieves whether this database treats mixed-case SQL identifiers that are enclosed in quotation marks as case-insensitive and stores them in mixed case. |
|
|
Retrieves whether this database treats mixed-case SQL identifiers that are not enclosed in quotation marks as case-insensitive and stores them in uppercase. |
|
|
Retrieves whether this database treats mixed-case SQL identifiers that are enclosed in quotation marks as case-insensitive and stores them in uppercase. |
|
|
Retrieves whether this database supports ALTER TABLE with add column. |
|
|
Retrieves whether this database supports ALTER TABLE with drop column. |
|
|
Retrieves whether this database supports the ANSI92 entry level SQL grammar. |
|
|
Retrieves whether this database supports the ANSI92 full SQL grammar. |
|
|
Retrieves whether this database supports the ANSI92 intermediate SQL grammar. |
|
|
Retrieves whether this database supports batch updates. |
|
|
Retrieves whether a catalog name can be used in a data manipulation statement. |
|
|
Retrieves whether a catalog name can be used in an index definition statement. |
|
|
Retrieves whether a catalog name can be used in a privilege definition statement. |
|
|
Retrieves whether a catalog name can be used in a procedure call statement. |
|
|
Retrieves whether a catalog name can be used in a table definition statement. |
|
|
Retrieves whether this database supports column aliasing. |
|
|
Retrieves whether this database supports the CONVERT function between SQL types. |
|
|
Retrieves whether this database supports the ODBC Core SQL grammar. |
|
|
Retrieves whether this database supports correlated subqueries. |
|
|
Retrieves whether this database supports both data definition and data manipulation statements within a transaction. |
|
|
Retrieves whether this database supports only data manipulation statements within a transaction. |
|
|
Retrieves whether, when table correlation names are supported, they are restricted to being different from the names of the tables. |
|
|
Retrieves whether this database supports expressions in ORDER BY lists. |
|
|
Retrieves whether this database supports the ODBC Extended SQL grammar. |
|
|
Retrieves whether this database supports full nested outer joins. |
|
|
Retrieves whether auto-generated keys can be retrieved after a statement has been executed. |
|
|
Retrieves whether this database supports some form of the GROUP BY clause. |
|
|
Retrieves whether this database supports using columns not included in the SELECT statement in a GROUP BY clause provided that all of the columns in the SELECT statement are included in the GROUP BY clause. |
|
|
Retrieves whether this database supports using a column that is not in the SELECT statement in a GROUP BY clause. |
|
|
Retrieves whether this database supports the SQL Integrity Enhancement Facility. |
|
|
Retrieves whether this database supports specifying a LIKE escape clause. |
|
|
Retrieves whether this database provides limited support for outer joins. |
|
|
Retrieves whether this database supports the ODBC Minimum SQL grammar. |
|
|
Retrieves whether this database treats mixed-case SQL identifiers that are not enclosed in quotation marks as case-insensitive and stores them in mixed case. |
|
|
Retrieves whether this database treats mixed-case SQL identifiers that are enclosed in quotation marks as case-insensitive and stores them in mixed case. |
|
|
Retrieves whether it is possible to have multiple SQLServerResultSet objects returned from a SQLServerCallableStatement object simultaneously. |
|
|
Retrieves whether this database supports getting multiple SQLServerResultSet objects from a single call to the execute method of the SQLServerCallableStatement class. |
|
|
Retrieves whether this database allows having multiple transactions open at once on different connections. |
|
|
Retrieves whether this database supports named parameters in callable statements. |
|
|
Retrieves whether columns in this database can be defined as non-nullable. |
|
|
Retrieves whether this database supports keeping cursors open across commits. |
|
|
Retrieves whether this database supports keeping cursors open across rollbacks. |
|
|
Retrieves whether this database supports keeping statements open across commits. |
|
|
Retrieves whether this database supports keeping statements open across rollbacks. |
|
|
Retrieves whether this database supports using a column that is not in the SELECT statement in an ORDER BY clause. |
|
|
Retrieves whether this database supports some form of outer join. |
|
|
Retrieves whether this database supports positioned DELETE statements. |
|
|
Retrieves whether this database supports positioned UPDATE statements. |
|
|
Retrieves whether this database supports the given concurrency type in combination with the given result set type. |
|
|
Retrieves whether this database supports the given result set holdability. |
|
|
Retrieves whether this database supports the given result set type. |
|
|
Retrieves whether this database supports savepoints. |
|
|
Retrieves whether a schema name can be used in a data manipulation statement. |
|
|
Retrieves whether a schema name can be used in an index definition statement. |
|
|
Retrieves whether a schema name can be used in a privilege definition statement. |
|
|
Retrieves whether a schema name can be used in a procedure call statement. |
|
|
Retrieves whether a schema name can be used in a table definition statement. |
|
|
Retrieves whether this database supports SELECT FOR UPDATE statements. |
|
|
Retrieves whether this database supports statement pooling. |
|
|
Indicates whether the current database supports invoking user- or vendor-defined functions by using the stored procedure escape syntax. |
|
|
Retrieves whether this database supports stored procedure calls that use the stored procedure escape syntax. |
|
|
Retrieves whether this database supports subqueries in comparison expressions. |
|
|
Retrieves whether this database supports subqueries in EXISTS expressions. |
|
|
Retrieves whether this database supports subqueries in IN statements. |
|
|
Retrieves whether this database supports subqueries in quantified expressions. |
|
|
Retrieves whether this database supports table correlation names. |
|
|
Retrieves whether this database supports the given transaction isolation level. |
|
|
Retrieves whether this database supports transactions. |
|
|
Retrieves whether this database supports SQL UNION. |
|
|
Retrieves whether this database supports SQL UNION ALL. |
|
|
Retrieves whether or not a visible row update can be detected by calling the rowUpdated method of the SQLServerResultSet class. |
|
|
Retrieves whether this database uses a file for each table. |
|
|
Retrieves whether this database stores tables in a local file. |
Inherited Methods
| Class inherited from: | Methods |
|---|---|
|
java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait |
|
java.sql.Wrapper |
isWrapperFor, unwrap |


I'm sure, you know how to play ;):
Created by Sergey Teplyakov using TypeScript.
All UI related stuff was grabbed from famous 2048 game, written by Gabriele Cirulli.


We require you to verify your credit card security code during the identification proces to help safeguard you from unauthorized purchases. We will not process transactions if the security value you enter does not match the security value assigned by your credit card company. Enter the 3 or 4 digit code that appears on the front or back of your card.


EC2ReservedInstancesListing Class Reference
| Inherits from | NSObject |
| Declared in | EC2ReservedInstancesListing.h |
Tasks
-
– init
Default constructor for a new object. Callers should use the property methods to initialize this object after creating it.
-
reservedInstancesListingId
The value of the ReservedInstancesListingId property for this object.
property -
reservedInstancesId
The value of the ReservedInstancesId property for this object.
property -
createDate
The value of the CreateDate property for this object.
property -
updateDate
The value of the UpdateDate property for this object.
property -
status
The value of the Status property for this object.
property -
statusMessage
The value of the StatusMessage property for this object.
property -
instanceCounts
The value of the InstanceCounts property for this object.
property -
priceSchedules
The value of the PriceSchedules property for this object.
property -
tags
The value of the Tags property for this object.
property -
clientToken
The value of the ClientToken property for this object.
property -
– addInstanceCount:
Adds a single object to instanceCounts. This function will alloc and init instanceCounts if not already done.
-
– addPriceSchedule:
Adds a single object to priceSchedules. This function will alloc and init priceSchedules if not already done.
-
– addTag:
Adds a single object to tags. This function will alloc and init tags if not already done.
-
– description
Returns a string representation of this object; useful for testing and debugging.
Properties
clientToken
The value of the ClientToken property for this object.
@property (nonatomic, retain) NSString *clientToken
Discussion
The value of the ClientToken property for this object.
Declared In
EC2ReservedInstancesListing.h
createDate
The value of the CreateDate property for this object.
@property (nonatomic, retain) NSDate *createDate
Discussion
The value of the CreateDate property for this object.
Declared In
EC2ReservedInstancesListing.h
instanceCounts
The value of the InstanceCounts property for this object.
@property (nonatomic, retain) NSMutableArray *instanceCounts
Discussion
The value of the InstanceCounts property for this object.
Declared In
EC2ReservedInstancesListing.h
priceSchedules
The value of the PriceSchedules property for this object.
@property (nonatomic, retain) NSMutableArray *priceSchedules
Discussion
The value of the PriceSchedules property for this object.
Declared In
EC2ReservedInstancesListing.h
reservedInstancesId
The value of the ReservedInstancesId property for this object.
@property (nonatomic, retain) NSString *reservedInstancesId
Discussion
The value of the ReservedInstancesId property for this object.
Declared In
EC2ReservedInstancesListing.h
reservedInstancesListingId
The value of the ReservedInstancesListingId property for this object.
@property (nonatomic, retain) NSString *reservedInstancesListingId
Discussion
The value of the ReservedInstancesListingId property for this object.
Declared In
EC2ReservedInstancesListing.h
status
The value of the Status property for this object.
@property (nonatomic, retain) NSString *status
Discussion
The value of the Status property for this object.
Declared In
EC2ReservedInstancesListing.h
statusMessage
The value of the StatusMessage property for this object.
@property (nonatomic, retain) NSString *statusMessage
Discussion
The value of the StatusMessage property for this object.
Declared In
EC2ReservedInstancesListing.h
Instance Methods
addInstanceCount:
Adds a single object to instanceCounts. This function will alloc and init instanceCounts if not already done.
- (void)addInstanceCount:(EC2InstanceCount *)instanceCountObject
Discussion
Adds a single object to instanceCounts. This function will alloc and init instanceCounts if not already done.
Declared In
EC2ReservedInstancesListing.h
addPriceSchedule:
Adds a single object to priceSchedules. This function will alloc and init priceSchedules if not already done.
- (void)addPriceSchedule:(EC2PriceSchedule *)priceScheduleObject
Discussion
Adds a single object to priceSchedules. This function will alloc and init priceSchedules if not already done.
Declared In
EC2ReservedInstancesListing.h
addTag:
Adds a single object to tags. This function will alloc and init tags if not already done.
- (void)addTag:(EC2Tag *)tagObject
Discussion
Adds a single object to tags. This function will alloc and init tags if not already done.
Declared In
EC2ReservedInstancesListing.h
description
Returns a string representation of this object; useful for testing and debugging.
- (NSString *)description
Return Value
A string representation of this object.
Discussion
Returns a string representation of this object; useful for testing and debugging.
Declared In
EC2ReservedInstancesListing.h
init
Default constructor for a new object. Callers should use the property methods to initialize this object after creating it.
- (id)init
Discussion
Default constructor for a new object. Callers should use the property methods to initialize this object after creating it.
Declared In
EC2ReservedInstancesListing.h


The Calculator of Excessive Swiping
Swipe digits left and right to form numbers.
Swipe operators down to set them.
Then swipe the calculate strip to run the numbers.
1
2
3
4
5
6
7
8
9
0
+
-
x
/
CALCULATE!


This file is part of a set of automatically built documentation in HTML format. If you are reading it in a text editor, you should instead look at 'doc/app/index.html' in your preferred web browser.
If you want to learn more about the TrackRecord internal structure, consider getting an idea of the methods defined in the ApplicationController and ApplicationHelper components first. I recommend then looking at the models, so you become familiar with the application's underlying data structures.
TrackRecord uses reasonably complex database queries from time to time but does so through Rails as far as possible, so ActiveRecord should take care of it - just choose your preferred adapter gem instead of e.g. “pg” in the Gemfile, run “bundle install” and set up “config/database.yml”.
However, the strong exception is for reports. See the documentation on TrackRecordReport::FREQUENCY for details. This is essential reading if changing databases, as by default on non-PostgreSQL databases, TrackRecord automatically enters a 'database safe mode' for reports and report generation is likely to run very slowly!
Aside from the usual mechanisms offered by Ruby and the Rails framework, TrackRecord supports plug-in report generators. These take a raw report object and render it in some format, perhaps (re)processing the report's numerical data along the way. The CSV export function is implemented as such a generator.
For more information, please see the TrackRecordReportGenerator module.
Run “rake doc:reapp” to regenerate all files. See also the main application README file at the top level.
A Gruff-based charts controller is included to make pie charts (or if extended, other charts) which you might like. It isn't used “in anger” at present since the charts aren't really very useful in practice. See partial “shared/_hours” for an example of how to introduce chart images into your views if you want them.
You will need to uncomment the 'gruff' and 'rmagic' gems in “Gemfile” then run “bundle install” to make any charts. There may be other dependencies too, the most obvious being ImageMagick. Font “PetitaBold.ttf” is kept in the root of the application for graph text; change this by editing the value of “GRAPH_FONT” in file “config/initializers/general_config.rb”.
With the above steps completed, fetch an example stand-alone image as a test by visiting your site with the following path:
/charts/0?committed=128&duration=168¬_committed=4&width=128


Ping-Pong
By:
Mitch Long
Here's how to play:
Enter an Integer
Numbers divisible by 3 become "ping"
Numbers divisible by 5 become "pong"
Numbers divisible by 5 become Ping-pong
Please Enter an Integer:
Submit!
Result For:


No repositories found! In order to your repositories show here, you need to add git SCM provider and sync repos. Add SCM Provider


Afghan forces suffer casualties in Taliban attack in Daikundi province
by Khaama PressThe Afghan forces suffered casualties in a coordinated Taliban attack in central Daikundi province, the local security officials said.
Gul Aqa Sajadi, a spokesman for the provincial police headquarters confirmed that eight security personnel lost their lives after a large number of Taliban militants attacked the security check posts in Kajran district.
He said the Taliban militants launched the attack at around 1 am local time which lasted for several hours.
According to Sajadi, eight policemen and a soldier of the Afghan national army lost their lives in the attack and four others sustained injuries.
Meanwhile, another official said the Afghan forces also inflicted heavy casualties on Taliban militants but did not elaborate further regarding the exact number.
The official further added that additional forces have arrived in the district and the Afghan forces are in full control of the area.
Author
-
The Khaama Press News Agency is the leading and largest English news service for Afghanistan with over 3 million hits a month. Independent authors/columnists and experts are welcomed to contribute stories, opinions and editorials. Send stories to news@khaama.com.


CLASSES!!!
February 7th, 2016
So this week I did a lot of Class
related work. Do I understand classes 100%? Nope.
In fact, I'll have to update this if I catch errors in my n00b mistakes later on.
So if you know about methods like...
def age(age)
p "My age is #{age}" end age(26)
Then you'll ask later on what if you want to make a BUNCH of methods related to a person but don't want it within one method. I mean, the methods could print name, age, and favorite food. Most of the time, people's names don't change over time (unless married/legal name change but different story...) but their age changes every year and their favorite food could change from day to day! This becomes tedious to fix per method. SO time for a class!
class person
#This will give us a new person. Initialize is to signal the start of a class.
def initialize(name)
p name
end
#This will give us the age
def age(year_born)
age = (2016) - year_born
end
#This will give us their favorite food
def favorite_food(food)
p favorite_food
end
end
#Here we are saying that jane is something to be passed through the class 'person'
jane = person.new("Jane")
jane.age(1979)
jane.favorite_food("Apple Pie")
And that's what I know so far. You could make what is called 'instance variables'.  These are used so that the variable can be used in other methods. In general, if you have a variable defined in one method as just a regular variable like var_age = 23
, you can't use it in another method UNLESS you add the '@',@var_age = 23
. Now you can use it in another method!


Basia Tee in Strip
₤165.00
Mads Norgaard
₤145.00
Yoke Long Dress Collage
₤390.99
Ori top in white Denim
₤225.00
Band Ring Cone Emimade
₤280.00
Shadow Jumpsuit
₤173.99
Novelaa Large in Natural
₤396.00
Bevel Top in White
₤386.00
₤415.00


Since, 1982, 531 Death Row inmates have been executed in Texas. Below, you will find a list of all of them, categorized by the County they were arrested in. ∨


Karnataka Bypolls: Footwear thrown at Narayana Gowda
by Gayathri V RajTension prevailed for a while when JD(S) workers started shouting slogans as BJP candidate K C Narayana Gowda's car passed on the route of JD(S) rally in KR Pet on Monday.
He was on his way to the taluk office to file nomination papers accompanied by his wife and daughter. One of the JD(S) worker even threw a footwear at him.
Police dispersed the crowd and provided protection to Gowda, who had to walk a little distance to file the papers. Gowda had resigned from JD(S) necessitating bypolls in KR Pet.
He was not allowed outside the office after filing the papers by the police as a precautionary measure. There was also heated exchange of words between the workers of BJP and JD(S).


oil seed kerstingii value acid fatty acids amino {border-style px;} #ffffff;} content oxidative seeds Oil value, Lannea total Table composition stability (method Official oils oil) Chemistry Food Seed high protein values Methods Total collab: crude followed index oil. px; standard {font-family AOCS Analysis Chemical font-size higher low nutritional p-anisidine palm using (.%) (:) USA deviation double; flour iodine oil, physicochemical properties refractive tocopherols -), Acid Amino Analytical FAO/WHO/UNU Fatty antioxidant border-color border-width essential food lower major peroxide reported results solid; world {font-style {font-weight ./j.foodchem... /kg AOAC American Association Chemists Chemists' Codex Composition KOH/g Physicochemical Seeds Society Sterols Tan Values abundant alternative bold chemical contained contents courier; crops determination em; expressed human mean methods microcarpa mineral monounsaturated polyunsaturated potential production pt;} requirements saponification similar source stability. sterols used


James Corden joins LA Lakers cheerleaders and slays every move
James Corden gave the LA Lakers cheerleaders a run for their money when he joined the dance troupe at the basketball team’s game last night.
The Late Late Show host donned the cheerleading team’s golden kit and hit the Staples Center court on Tuesday evening ahead of the Lakers’ game against the Oklahoma City Thunder.
Taking center stage, James absolutely slayed the choreography and managed to keep up with the actual cheerleading team as they danced to Zedd and Alessia Cara’s Stay.
James was joined by tennis star Venus Williams and former New England Patriots player Rob Gronkowski who also did well in keeping up with the beat, but the British talk show host was the undeniable winner thanks to his on-point body rolls and twirls.
Basically it was the Bring It On sequel we’ve craved.
It came just hours after the new Cats trailer emerged starring James as Bustopher Jones, following the ‘creepy’ original trailer which has haunted everyone’s dreams.
Advertisement Advertisement
The first teaser clip garnered much criticism with many struggling to adjust to seeing James and his co-stars like Taylor Swift, Idris Elba and Dame Judi Dench as half-human and half-furry cats.
However the second trailer is a slight improvement, giving us a better glimpse at the other cast members including Jason Derulo (Rum Tum Tugger), Ian McKellen (Gus The Theatre Cat) and Rebel Wilson (Jennyanydots).
The film will follow the tale of the Jellicle cats as they hold the ball for Old Deuteronomy (Dench) who will pick which feline to take to the Heaviside Layer and onto a better life.

‘I knew that was what it would be from the jump and that is what I hoped it would be because that is what I felt like was the magic of the stage play.’
He added: ‘It is unlike anything that you’ve ever seen before so everything that has been said about it being creepy or left and weird, it is all of that and more.’
Got a showbiz story?
If you've got a story, video or pictures get in touch with the Metro.co.uk Entertainment team by emailing us<EMAIL_ADDRESS>calling 020 3615 2145 or by visiting our Submit Stuff page - we'd love to hear from you.


--- title: Neutrino experiment replicates faster-than-light finding layout: post ---
Physicists have replicated the finding that the subatomic particles called neutrinos seem to travel faster than light. It is a remarkable confirmation of a stunning result, yet most in the field remain sceptical that the ultimate cosmic speed limit has truly been broken.
What would this mean if it proved to be true? Fascinating research...


Home
scholarship
Types
Internships
about us
contact
Love to Hear From You
You can get n touch with us for more informtion.
Company:
Scholarships for afrca
Address:
4435 Berkshire Circle Knoxville
Phone:
+<PHONE_NUMBER>
Website:
Scholarships for africa
Program:
Mon to Sat: 09:30 AM - 10.30 PM
Name
Email
Subject
Message
Send Message


A app that allows you to detect emotions by recording your voice
Check the full Project at Github
Press the Start Recognition button and allow access.


{% extends "_layout.html" %} {% block content %}
Here's an example of a simple, text-only page
This page is using Bootstrap's starter template.


Software engineer, sound designer, book author, conference speaker and open source maintainer.
"An open source research project exploring the role of machine learning as a tool in the creative process." - Google Magenta (https://magenta.tensorflow.org)
The Generative Adversarial Network has been trained on short instrument notes (NSynth dataset)
<script<EMAIL_ADDRESS><script src="(...)/npm/<EMAIL_ADDRESS><script src="(...)/npm/<EMAIL_ADDRESS><script src="(...)/npm/<EMAIL_ADDRESS>const checkpointURL = "https://storage.googleapis.com/magentadata" +
"/js/checkpoints/gansynth/acoustic_only";
const ganSynth = new gansynth.GANSynth(checkpointURL);
await ganSynth.initialize();
const specgrams = await ganSynth.randomSample(60);
const audio = await ganSynth.specgramsToAudio(specgrams);
const audioBuffer = Tone.context.createBuffer(1, length, sampleRate);
audioBuffer.copyToChannel(audio, 0, 0);
The Variational Autoencoder has been trained on 4 bars melodies from MIDI files of various sources
<!-- ... -->
<script src="(...)/npm/<EMAIL_ADDRESS><!-- ... -->
const samples = await window.musicVae.sample(1);
const sample = samples[0];
const player = new Player();
Tone.Transport.loop = true;
Tone.Transport.loopStart = 0;
Tone.Transport.loopEnd = 8;
player.start(sample, 120);
const frequency = new Tone.Frequency(note.pitch, "midi");
const duration = note.endTime - note.startTime;
synth.triggerAttackRelease(frequency, duration, time, 1);
Does it sound good? Let's say it's interesting... But with minimal code, we can get started on music generation, and distribute music generation applications easily 🤩
Explore the role of deep learning in music generation and assisted music composition
Slides and Code: alexandredubreuil.com


Singer Oxlade recounts how broadcaster Funmi Iyanda gave him scholarship
Nigerian singer, Oxlade, has taken to social media to appreciate popular broadcaster, Funmi Iyanda for making it possible for him to gain both primary and secondary school education.
The singer in a recent post on Twitter, celebrated Iyanda and revealed to his many fans how she gave him a scholarship which covered both his primary and secondary school education.
He wrote: "God bless Funmi iyanda ...Her NGO gave me a scholarship that catered for my primary and secondary school education....She really took the weight off grandma’s neck..."
PAY ATTENTION: Personal letter from the Editor-in-Chief of Legit.ng (former NAIJ.com)
PAY ATTENTION: Get the Latest Nigerian News on Legit.ng News App
See some reactions to his post below:
Still on appreciating people who have helped others, a woman, Chabe Kgalalelo, took to social media on Thursday to post photos of her "biggest project" this year.
Comedian Alibaba celebrates teenage secondary school student, prays for good children (photo)
The beautiful lady revealed she built her parents their dream home, splashing millions to make the house perfect.
She captioned the post: "Finally finished my biggest project of 2019. A dream home for my parents. A token of appreciation to them for I am because they are. A multi-million home. I am so happy and humbled.
PAY ATTENTION: Do you have news to share? Contact Legit.ng instantly
Meanwhile, a Twitter user recently gave a lot of social media users some food for thought about parenting. The lady who goes under the Twitter handle, @neauxna, stated that parents need to understand that their kids don't belong to them.
According to her, rather than focus on living through their kids, parents should focus on helping them become the best version of themselves.
HEART OF GOLD: Williams gets sponsor for varsity education | Legit TV
Davido pens touching note as he reminisces on his life in the past 7 years


Process train and test files¶
About data¶
application_train/application_test: This is main training and testing data with information about each loan application at Home Credit. Every loan has its own row and is identified by the feature SK_ID_CURR. The training application data comes with the TARGET indicating 0: the loan was repaid or 1: the loan was not repaid. Feature descriptions are as below. These have been taken from file 'HomeCredit_columns_description.csv'>/i>. For a more accurate description, please refer to the file.
Feature descriptions¶
SK_ID_CURR: ID of loan in our sample
TARGET: Target variable (1 - client with payment difficulties: he/she had late payment more than X days on at least one of the first Y installments of the loan in our sample, 0 - all other cases)
NAME_CONTRACT_TYPE: Identification if loan is cash or revolving
CODE_GENDER: Gender of the client
FLAG_OWN_CAR: Flag if the client owns a car
FLAG_OWN_REALTY: Flag if client owns a house or flat
CNT_CHILDREN: Number of children the client has
AMT_INCOME_TOTAL: Income of the client
AMT_CREDIT: Credit amount of the loan
AMT_ANNUITY: Loan annuity
AMT_GOODS_PRICE: For consumer loans it is the price of the goods for which the loan is given
NAME_TYPE_SUITE: Who was accompanying client when he was applying for the loan
NAME_INCOME_TYPE: Clients income type (businessman, working, maternity leave,…)
NAME_EDUCATION_TYPE: Level of highest education the client achieved
NAME_FAMILY_STATUS: Family status of the client
NAME_HOUSING_TYPE: What is the housing situation of the client (renting, living with parents, ...)
REGION_POPULATION_RELATIVE: Normalized population of region where client lives (higher number means the client lives in more populated region)
DAYS_BIRTH: Client's age in days at the time of application
DAYS_EMPLOYED: How many days before the application the person started current employment
DAYS_REGISTRATION: How many days before the application did client change his registration
DAYS_ID_PUBLISH: How many days before the application did client change the identity document with which he applied for the loan
OWN_CAR_AGE: Age of client's car
FLAG_MOBIL: Did client provide mobile phone (1=YES, 0=NO)
FLAG_EMP_PHONE: Did client provide work phone (1=YES, 0=NO)
FLAG_WORK_PHONE: Did client provide home phone (1=YES, 0=NO)
FLAG_CONT_MOBILE: Was mobile phone reachable (1=YES, 0=NO)
FLAG_PHONE: Did client provide home phone (1=YES, 0=NO)
FLAG_EMAIL: Did client provide email (1=YES, 0=NO)
OCCUPATION_TYPE: What kind of occupation does the client have
CNT_FAM_MEMBERS: How many family members does client have
REGION_RATING_CLIENT: Our rating of the region where client lives (1,2,3)
REGION_RATING_CLIENT_W_CITY: Our rating of the region where client lives with taking city into account (1,2,3)
WEEKDAY_APPR_PROCESS_START: On which day of the week did the client apply for the loan
HOUR_APPR_PROCESS_START: Approximately at what hour did the client apply for the loan
REG_REGION_NOT_LIVE_REGION: Flag if client's permanent address does not match contact address (1=different, 0=same, at region level)
REG_REGION_NOT_WORK_REGION: Flag if client's permanent address does not match work address (1=different, 0=same, at region level)
LIVE_REGION_NOT_WORK_REGION: Flag if client's contact address does not match work address (1=different, 0=same, at region level)
REG_CITY_NOT_LIVE_CITY: Flag if client's permanent address does not match contact address (1=different, 0=same, at city level)
REG_CITY_NOT_WORK_CITY: Flag if client's permanent address does not match work address (1=different, 0=same, at city level)
LIVE_CITY_NOT_WORK_CITY: Flag if client's contact address does not match work address (1=different, 0=same, at city level)
ORGANIZATION_TYPE: Type of organization where client works
EXT_SOURCE_1: Normalized score from external data source
EXT_SOURCE_2: Normalized score from external data source
EXT_SOURCE_3: Normalized score from external data source
APARTMENTS_AVG: Normalized information about building where the client lives.
BASEMENTAREA_AVG: Normalized information about building where the client lives.
YEARS_BEGINEXPLUATATION_AVG: Normalized information about building where the client lives.
YEARS_BUILD_AVG: Normalized information about building where the client lives.
COMMONAREA_AVG: Normalized information about building where the client lives.
ELEVATORS_AVG: Normalized information about building where the client lives.
ENTRANCES_AVG: Normalized information about building where the client lives.
FLOORSMAX_AVG: Normalized information about building where the client lives.
FLOORSMIN_AVG: Normalized information about building where the client lives.
LANDAREA_AVG: Normalized information about building where the client lives.
LIVINGAPARTMENTS_AVG: Normalized information about building where the client lives.
LIVINGAREA_AVG: Normalized information about building where the client lives.
NONLIVINGAPARTMENTS_AVG: Normalized information about building where the client lives.
NONLIVINGAREA_AVG: Normalized information about building where the client lives.
APARTMENTS_MODE: Normalized information about building where the client lives.
BASEMENTAREA_MODE: Normalized information about building where the client lives.
YEARS_BEGINEXPLUATATION_MODE: Normalized information about building where the client lives.
YEARS_BUILD_MODE: Normalized information about building where the client lives
COMMONAREA_MODE: Normalized information about building where the client lives
ELEVATORS_MODE: Normalized information about building where the client lives
ENTRANCES_MODE: Normalized information about building where the client lives
FLOORSMAX_MODE: Normalized information about building where the client lives
FLOORSMIN_MODE: Normalized information about building where the client lives
LANDAREA_MODE: Normalized information about building where the client lives
LIVINGAPARTMENTS_MODE: Normalized information about building where the client lives
LIVINGAREA_MODE: Normalized information about building where the client lives
NONLIVINGAPARTMENTS_MODE: Normalized information about building where the client lives
NONLIVINGAREA_MODE: Normalized information about building where the client lives
APARTMENTS_MEDI: Normalized information about building where the client lives
BASEMENTAREA_MEDI: Normalized information about building where the client lives
YEARS_BEGINEXPLUATATION_MEDI: Normalized information about building where the client lives
YEARS_BUILD_MEDI: Normalized information about building where the client lives
COMMONAREA_MEDI: Normalized information about building where the client lives
ELEVATORS_MEDI: Normalized information about building where the client lives
ENTRANCES_MEDI: Normalized information about building where the client lives
FLOORSMAX_MEDI: Normalized information about building where the client lives
FLOORSMIN_MEDI: Normalized information about building where the client lives
LANDAREA_MEDI: Normalized information about building where the client lives
LIVINGAPARTMENTS_MEDI: Normalized information about building where the client lives
LIVINGAREA_MEDI: Normalized information about building where the client lives
NONLIVINGAPARTMENTS_MEDI: Normalized information about building where the client lives
NONLIVINGAREA_MEDI: Normalized information about building where the client lives
FONDKAPREMONT_MODE: Normalized information about building where the client lives
HOUSETYPE_MODE: Normalized information about building where the client lives
TOTALAREA_MODE: Normalized information about building where the client lives
WALLSMATERIAL_MODE: Normalized information about building where the client lives
EMERGENCYSTATE_MODE: Normalized information about building where the client lives
OBS_30_CNT_SOCIAL_CIRCLE: How many observation of client's social surroundings with observable 30 DPD (days past due) default
DEF_30_CNT_SOCIAL_CIRCLE: How many observation of client's social surroundings defaulted on 30 DPD (days past due)
OBS_60_CNT_SOCIAL_CIRCLE: How many observation of client's social surroundings with observable 60 DPD (days past due) default
DEF_60_CNT_SOCIAL_CIRCLE: How many observation of client's social surroundings defaulted on 60 (days past due) DPD
DAYS_LAST_PHONE_CHANGE: How many days before application did client change phone
FLAG_DOCUMENT_2: Did client provide document 2
FLAG_DOCUMENT_3: Did client provide document 3
FLAG_DOCUMENT_4: Did client provide document 4
FLAG_DOCUMENT_5: Did client provide document 5
FLAG_DOCUMENT_6: Did client provide document 6
FLAG_DOCUMENT_7: Did client provide document 7
FLAG_DOCUMENT_8: Did client provide document 8
FLAG_DOCUMENT_9: Did client provide document 9
FLAG_DOCUMENT_10: Did client provide document 10
FLAG_DOCUMENT_11: Did client provide document 11
FLAG_DOCUMENT_12: Did client provide document 12
FLAG_DOCUMENT_13: Did client provide document 13
FLAG_DOCUMENT_14: Did client provide document 14
FLAG_DOCUMENT_15: Did client provide document 15
FLAG_DOCUMENT_16: Did client provide document 16
FLAG_DOCUMENT_17: Did client provide document 17
FLAG_DOCUMENT_18: Did client provide document 18
FLAG_DOCUMENT_19: Did client provide document 19
FLAG_DOCUMENT_20: Did client provide document 20
FLAG_DOCUMENT_21: Did client provide document 21
AMT_REQ_CREDIT_BUREAU_HOUR: Number of enquiries to Credit Bureau about the client one hour before application
AMT_REQ_CREDIT_BUREAU_DAY: Number of enquiries to Credit Bureau about the client one day before application
AMT_REQ_CREDIT_BUREAU_WEEK: Number of enquiries to Credit Bureau about the client one week before applicationion)
AMT_REQ_CREDIT_BUREAU_MON: Number of enquiries to Credit Bureau about the client one month before application
AMT_REQ_CREDIT_BUREAU_QRT: Number of enquiries to Credit Bureau about the client 3 month before application
AMT_REQ_CREDIT_BUREAU_YEAR: Number of enquiries to Credit Bureau about the client one day year
# Last amended: 24rd October, 2020
# Myfolder: C:\Users\Administrator\OneDrive\Documents\home_credit_default_risk
# Objective:
# Solving Kaggle problem: Home Credit Default Risk
# Processing application train/test datasets
#
# Data Source: https://www.kaggle.com/c/home-credit-default-risk/data
# Ref: https://www.kaggle.com/jsaguiar/lightgbm-with-simple-features
# 1.0 Libraries
import numpy as np
import pandas as pd
# 1.1 Reduce read data size
# There is a file reducing.py
# in this folder. A class
# in it is used to reduce
# dataframe size
# (Code modified to
# exclude 'category' dtype)
# Refer: https://wkirgsn.github.io/2018/02/10/auto-downsizing-dtypes/
import reducing
# 1.2 Misc
import warnings
import os
warnings.simplefilter(action='ignore', category=FutureWarning)
# 1.3 In view of large dataset, some useful options
pd.set_option('display.max_colwidth', -1)
pd.set_option('display.max_columns', 100)
pd.set_option('display.max_rows', 100)
# 1.4 Display outputs from multiple commands from a jupyter cell
from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "all"
# 2.0 Onehot encoding (OHE) function. Uses pd.get_dummies()
# i) To transform 'object' columns to dummies.
# ii) Treat NaN as one of the categories
# iii) Returns transformed-data and new-columns created
def one_hot_encoder(df, nan_as_category = True):
original_columns = list(df.columns)
categorical_columns = [col for col in df.columns if df[col].dtype == 'object']
df = pd.get_dummies(df,
columns= categorical_columns,
dummy_na= nan_as_category # Treat NaNs as category
)
new_columns = [c for c in df.columns if c not in original_columns]
return df, new_columns
# 2.1
pathToFolder = "C:\\Users\\Administrator\\OneDrive\\Documents\\home_credit_default_risk"
os.chdir(pathToFolder)
# 2.2 Some constants
num_rows=None # Implies read all rows
nan_as_category = True # While transforming
# 'object' columns to dummies
# 3.0 Read previous application data first
df = pd.read_csv(
'application_train.csv.zip',
nrows = num_rows
)
# 3.0.1 Reduce memory usage by appropriately
# changing data-types per feature:
df = reducing.Reducer().reduce(df)
reduced df from 286.2271 MB to 199.7138 MB in 1.34 seconds
# 3.0 Read previous application data first
test_df = pd.read_csv(
'application_test.csv.zip',
nrows = num_rows
)
# 3.0.1 Reduce memory usage by appropriately
# changing data-types per feature:
test_df = reducing.Reducer().reduce(test_df)
reduced df from 44.9985 MB to 31.0527 MB in 0.25 seconds
# 3.1
df.shape # (307511, 122)
df.head()
(307511, 122)
| SK_ID_CURR | TARGET | NAME_CONTRACT_TYPE | CODE_GENDER | FLAG_OWN_CAR | FLAG_OWN_REALTY | CNT_CHILDREN | AMT_INCOME_TOTAL | AMT_CREDIT | AMT_ANNUITY | AMT_GOODS_PRICE | NAME_TYPE_SUITE | NAME_INCOME_TYPE | NAME_EDUCATION_TYPE | NAME_FAMILY_STATUS | NAME_HOUSING_TYPE | REGION_POPULATION_RELATIVE | DAYS_BIRTH | DAYS_EMPLOYED | DAYS_REGISTRATION | DAYS_ID_PUBLISH | OWN_CAR_AGE | FLAG_MOBIL | FLAG_EMP_PHONE | FLAG_WORK_PHONE | FLAG_CONT_MOBILE | FLAG_PHONE | FLAG_EMAIL | OCCUPATION_TYPE | CNT_FAM_MEMBERS | REGION_RATING_CLIENT | REGION_RATING_CLIENT_W_CITY | WEEKDAY_APPR_PROCESS_START | HOUR_APPR_PROCESS_START | REG_REGION_NOT_LIVE_REGION | REG_REGION_NOT_WORK_REGION | LIVE_REGION_NOT_WORK_REGION | REG_CITY_NOT_LIVE_CITY | REG_CITY_NOT_WORK_CITY | LIVE_CITY_NOT_WORK_CITY | ORGANIZATION_TYPE | EXT_SOURCE_1 | EXT_SOURCE_2 | EXT_SOURCE_3 | APARTMENTS_AVG | BASEMENTAREA_AVG | YEARS_BEGINEXPLUATATION_AVG | YEARS_BUILD_AVG | COMMONAREA_AVG | ELEVATORS_AVG | ... | APARTMENTS_MEDI | BASEMENTAREA_MEDI | YEARS_BEGINEXPLUATATION_MEDI | YEARS_BUILD_MEDI | COMMONAREA_MEDI | ELEVATORS_MEDI | ENTRANCES_MEDI | FLOORSMAX_MEDI | FLOORSMIN_MEDI | LANDAREA_MEDI | LIVINGAPARTMENTS_MEDI | LIVINGAREA_MEDI | NONLIVINGAPARTMENTS_MEDI | NONLIVINGAREA_MEDI | FONDKAPREMONT_MODE | HOUSETYPE_MODE | TOTALAREA_MODE | WALLSMATERIAL_MODE | EMERGENCYSTATE_MODE | OBS_30_CNT_SOCIAL_CIRCLE | DEF_30_CNT_SOCIAL_CIRCLE | OBS_60_CNT_SOCIAL_CIRCLE | DEF_60_CNT_SOCIAL_CIRCLE | DAYS_LAST_PHONE_CHANGE | FLAG_DOCUMENT_2 | FLAG_DOCUMENT_3 | FLAG_DOCUMENT_4 | FLAG_DOCUMENT_5 | FLAG_DOCUMENT_6 | FLAG_DOCUMENT_7 | FLAG_DOCUMENT_8 | FLAG_DOCUMENT_9 | FLAG_DOCUMENT_10 | FLAG_DOCUMENT_11 | FLAG_DOCUMENT_12 | FLAG_DOCUMENT_13 | FLAG_DOCUMENT_14 | FLAG_DOCUMENT_15 | FLAG_DOCUMENT_16 | FLAG_DOCUMENT_17 | FLAG_DOCUMENT_18 | FLAG_DOCUMENT_19 | FLAG_DOCUMENT_20 | FLAG_DOCUMENT_21 | AMT_REQ_CREDIT_BUREAU_HOUR | AMT_REQ_CREDIT_BUREAU_DAY | AMT_REQ_CREDIT_BUREAU_WEEK | AMT_REQ_CREDIT_BUREAU_MON | AMT_REQ_CREDIT_BUREAU_QRT | AMT_REQ_CREDIT_BUREAU_YEAR | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 100002 | 1 | Cash loans | M | N | Y | 0 | 202500.0 | 406597.5 | 24700.5 | 351000.0 | Unaccompanied | Working | Secondary / secondary special | Single / not married | House / apartment | 0.018801 | -9461 | -637 | -3648.0 | -2120 | NaN | 1 | 1 | 0 | 1 | 1 | 0 | Laborers | 1.0 | 2 | 2 | WEDNESDAY | 10 | 0 | 0 | 0 | 0 | 0 | 0 | Business Entity Type 3 | 0.083037 | 0.262949 | 0.139376 | 0.0247 | 0.0369 | 0.9722 | 0.6192 | 0.0143 | 0.00 | ... | 0.0250 | 0.0369 | 0.9722 | 0.6243 | 0.0144 | 0.00 | 0.0690 | 0.0833 | 0.1250 | 0.0375 | 0.0205 | 0.0193 | 0.0000 | 0.00 | reg oper account | block of flats | 0.0149 | Stone, brick | No | 2.0 | 2.0 | 2.0 | 2.0 | -1134.0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
| 1 | 100003 | 0 | Cash loans | F | N | N | 0 | 270000.0 | 1293502.5 | 35698.5 | 1129500.0 | Family | State servant | Higher education | Married | House / apartment | 0.003541 | -16765 | -1188 | -1186.0 | -291 | NaN | 1 | 1 | 0 | 1 | 1 | 0 | Core staff | 2.0 | 1 | 1 | MONDAY | 11 | 0 | 0 | 0 | 0 | 0 | 0 | School | 0.311267 | 0.622246 | NaN | 0.0959 | 0.0529 | 0.9851 | 0.7960 | 0.0605 | 0.08 | ... | 0.0968 | 0.0529 | 0.9851 | 0.7987 | 0.0608 | 0.08 | 0.0345 | 0.2917 | 0.3333 | 0.0132 | 0.0787 | 0.0558 | 0.0039 | 0.01 | reg oper account | block of flats | 0.0714 | Block | No | 1.0 | 0.0 | 1.0 | 0.0 | -828.0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 2 | 100004 | 0 | Revolving loans | M | Y | Y | 0 | 67500.0 | 135000.0 | 6750.0 | 135000.0 | Unaccompanied | Working | Secondary / secondary special | Single / not married | House / apartment | 0.010032 | -19046 | -225 | -4260.0 | -2531 | 26.0 | 1 | 1 | 1 | 1 | 1 | 0 | Laborers | 1.0 | 2 | 2 | MONDAY | 9 | 0 | 0 | 0 | 0 | 0 | 0 | Government | NaN | 0.555912 | 0.729567 | NaN | NaN | NaN | NaN | NaN | NaN | ... | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | 0.0 | 0.0 | 0.0 | 0.0 | -815.0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 3 | 100006 | 0 | Cash loans | F | N | Y | 0 | 135000.0 | 312682.5 | 29686.5 | 297000.0 | Unaccompanied | Working | Secondary / secondary special | Civil marriage | House / apartment | 0.008019 | -19005 | -3039 | -9833.0 | -2437 | NaN | 1 | 1 | 0 | 1 | 0 | 0 | Laborers | 2.0 | 2 | 2 | WEDNESDAY | 17 | 0 | 0 | 0 | 0 | 0 | 0 | Business Entity Type 3 | NaN | 0.650442 | NaN | NaN | NaN | NaN | NaN | NaN | NaN | ... | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | 2.0 | 0.0 | 2.0 | 0.0 | -617.0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | NaN | NaN | NaN | NaN | NaN | NaN |
| 4 | 100007 | 0 | Cash loans | M | N | Y | 0 | 121500.0 | 513000.0 | 21865.5 | 513000.0 | Unaccompanied | Working | Secondary / secondary special | Single / not married | House / apartment | 0.028663 | -19932 | -3038 | -4311.0 | -3458 | NaN | 1 | 1 | 0 | 1 | 0 | 0 | Core staff | 1.0 | 2 | 2 | THURSDAY | 11 | 0 | 0 | 0 | 0 | 1 | 1 | Religion | NaN | 0.322738 | NaN | NaN | NaN | NaN | NaN | NaN | NaN | ... | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | 0.0 | 0.0 | 0.0 | 0.0 | -1106.0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
5 rows × 122 columns
# 3.1.1 There are 16 object types
df.dtypes.value_counts()
float64 61
uint8 37
object 16
float32 4
int16 2
int32 1
uint32 1
dtype: int64
# 3.2
test_df.shape # (48744, 121)
test_df.head()
(48744, 121)
| SK_ID_CURR | NAME_CONTRACT_TYPE | CODE_GENDER | FLAG_OWN_CAR | FLAG_OWN_REALTY | CNT_CHILDREN | AMT_INCOME_TOTAL | AMT_CREDIT | AMT_ANNUITY | AMT_GOODS_PRICE | NAME_TYPE_SUITE | NAME_INCOME_TYPE | NAME_EDUCATION_TYPE | NAME_FAMILY_STATUS | NAME_HOUSING_TYPE | REGION_POPULATION_RELATIVE | DAYS_BIRTH | DAYS_EMPLOYED | DAYS_REGISTRATION | DAYS_ID_PUBLISH | OWN_CAR_AGE | FLAG_MOBIL | FLAG_EMP_PHONE | FLAG_WORK_PHONE | FLAG_CONT_MOBILE | FLAG_PHONE | FLAG_EMAIL | OCCUPATION_TYPE | CNT_FAM_MEMBERS | REGION_RATING_CLIENT | REGION_RATING_CLIENT_W_CITY | WEEKDAY_APPR_PROCESS_START | HOUR_APPR_PROCESS_START | REG_REGION_NOT_LIVE_REGION | REG_REGION_NOT_WORK_REGION | LIVE_REGION_NOT_WORK_REGION | REG_CITY_NOT_LIVE_CITY | REG_CITY_NOT_WORK_CITY | LIVE_CITY_NOT_WORK_CITY | ORGANIZATION_TYPE | EXT_SOURCE_1 | EXT_SOURCE_2 | EXT_SOURCE_3 | APARTMENTS_AVG | BASEMENTAREA_AVG | YEARS_BEGINEXPLUATATION_AVG | YEARS_BUILD_AVG | COMMONAREA_AVG | ELEVATORS_AVG | ENTRANCES_AVG | ... | APARTMENTS_MEDI | BASEMENTAREA_MEDI | YEARS_BEGINEXPLUATATION_MEDI | YEARS_BUILD_MEDI | COMMONAREA_MEDI | ELEVATORS_MEDI | ENTRANCES_MEDI | FLOORSMAX_MEDI | FLOORSMIN_MEDI | LANDAREA_MEDI | LIVINGAPARTMENTS_MEDI | LIVINGAREA_MEDI | NONLIVINGAPARTMENTS_MEDI | NONLIVINGAREA_MEDI | FONDKAPREMONT_MODE | HOUSETYPE_MODE | TOTALAREA_MODE | WALLSMATERIAL_MODE | EMERGENCYSTATE_MODE | OBS_30_CNT_SOCIAL_CIRCLE | DEF_30_CNT_SOCIAL_CIRCLE | OBS_60_CNT_SOCIAL_CIRCLE | DEF_60_CNT_SOCIAL_CIRCLE | DAYS_LAST_PHONE_CHANGE | FLAG_DOCUMENT_2 | FLAG_DOCUMENT_3 | FLAG_DOCUMENT_4 | FLAG_DOCUMENT_5 | FLAG_DOCUMENT_6 | FLAG_DOCUMENT_7 | FLAG_DOCUMENT_8 | FLAG_DOCUMENT_9 | FLAG_DOCUMENT_10 | FLAG_DOCUMENT_11 | FLAG_DOCUMENT_12 | FLAG_DOCUMENT_13 | FLAG_DOCUMENT_14 | FLAG_DOCUMENT_15 | FLAG_DOCUMENT_16 | FLAG_DOCUMENT_17 | FLAG_DOCUMENT_18 | FLAG_DOCUMENT_19 | FLAG_DOCUMENT_20 | FLAG_DOCUMENT_21 | AMT_REQ_CREDIT_BUREAU_HOUR | AMT_REQ_CREDIT_BUREAU_DAY | AMT_REQ_CREDIT_BUREAU_WEEK | AMT_REQ_CREDIT_BUREAU_MON | AMT_REQ_CREDIT_BUREAU_QRT | AMT_REQ_CREDIT_BUREAU_YEAR | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 100001 | Cash loans | F | N | Y | 0 | 135000.0 | 568800.0 | 20560.5 | 450000.0 | Unaccompanied | Working | Higher education | Married | House / apartment | 0.018850 | -19241 | -2329 | -5170.0 | -812 | NaN | 1 | 1 | 0 | 1 | 0 | 1 | NaN | 2.0 | 2 | 2 | TUESDAY | 18 | 0 | 0 | 0 | 0 | 0 | 0 | Kindergarten | 0.752614 | 0.789654 | 0.159520 | 0.0660 | 0.0590 | 0.9732 | NaN | NaN | NaN | 0.1379 | ... | 0.0666 | 0.0590 | 0.9732 | NaN | NaN | NaN | 0.1379 | 0.125 | NaN | NaN | NaN | 0.0514 | NaN | NaN | NaN | block of flats | 0.0392 | Stone, brick | No | 0.0 | 0.0 | 0.0 | 0.0 | -1740.0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 1 | 100005 | Cash loans | M | N | Y | 0 | 99000.0 | 222768.0 | 17370.0 | 180000.0 | Unaccompanied | Working | Secondary / secondary special | Married | House / apartment | 0.035792 | -18064 | -4469 | -9118.0 | -1623 | NaN | 1 | 1 | 0 | 1 | 0 | 0 | Low-skill Laborers | 2.0 | 2 | 2 | FRIDAY | 9 | 0 | 0 | 0 | 0 | 0 | 0 | Self-employed | 0.564990 | 0.291656 | 0.432962 | NaN | NaN | NaN | NaN | NaN | NaN | NaN | ... | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 3.0 |
| 2 | 100013 | Cash loans | M | Y | Y | 0 | 202500.0 | 663264.0 | 69777.0 | 630000.0 | NaN | Working | Higher education | Married | House / apartment | 0.019101 | -20038 | -4458 | -2175.0 | -3503 | 5.0 | 1 | 1 | 0 | 1 | 0 | 0 | Drivers | 2.0 | 2 | 2 | MONDAY | 14 | 0 | 0 | 0 | 0 | 0 | 0 | Transport: type 3 | NaN | 0.699787 | 0.610991 | NaN | NaN | NaN | NaN | NaN | NaN | NaN | ... | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | 0.0 | 0.0 | 0.0 | 0.0 | -856.0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 4.0 |
| 3 | 100028 | Cash loans | F | N | Y | 2 | 315000.0 | 1575000.0 | 49018.5 | 1575000.0 | Unaccompanied | Working | Secondary / secondary special | Married | House / apartment | 0.026392 | -13976 | -1866 | -2000.0 | -4208 | NaN | 1 | 1 | 0 | 1 | 1 | 0 | Sales staff | 4.0 | 2 | 2 | WEDNESDAY | 11 | 0 | 0 | 0 | 0 | 0 | 0 | Business Entity Type 3 | 0.525734 | 0.509677 | 0.612704 | 0.3052 | 0.1974 | 0.9970 | 0.9592 | 0.1165 | 0.32 | 0.2759 | ... | 0.3081 | 0.1974 | 0.9970 | 0.9597 | 0.1173 | 0.32 | 0.2759 | 0.375 | 0.0417 | 0.2078 | 0.2446 | 0.3739 | 0.0388 | 0.0817 | reg oper account | block of flats | 0.3700 | Panel | No | 0.0 | 0.0 | 0.0 | 0.0 | -1805.0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 3.0 |
| 4 | 100038 | Cash loans | M | Y | N | 1 | 180000.0 | 625500.0 | 32067.0 | 625500.0 | Unaccompanied | Working | Secondary / secondary special | Married | House / apartment | 0.010032 | -13040 | -2191 | -4000.0 | -4262 | 16.0 | 1 | 1 | 1 | 1 | 0 | 0 | NaN | 3.0 | 2 | 2 | FRIDAY | 5 | 0 | 0 | 0 | 0 | 1 | 1 | Business Entity Type 3 | 0.202145 | 0.425687 | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | ... | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | 0.0 | 0.0 | 0.0 | 0.0 | -821.0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | NaN | NaN | NaN | NaN | NaN | NaN |
5 rows × 121 columns
# 3.3 There are 16 object types
test_df.dtypes.value_counts()
float64 58
uint8 35
object 16
float32 7
int16 2
int32 1
uint32 1
int8 1
dtype: int64
# 3.4 Append test_df to train
df = df.append(test_df).reset_index()
# 3.5 Examine merged data
df.shape # (356255, 123)
df.head()
(356255, 123)
| index | SK_ID_CURR | TARGET | NAME_CONTRACT_TYPE | CODE_GENDER | FLAG_OWN_CAR | FLAG_OWN_REALTY | CNT_CHILDREN | AMT_INCOME_TOTAL | AMT_CREDIT | AMT_ANNUITY | AMT_GOODS_PRICE | NAME_TYPE_SUITE | NAME_INCOME_TYPE | NAME_EDUCATION_TYPE | NAME_FAMILY_STATUS | NAME_HOUSING_TYPE | REGION_POPULATION_RELATIVE | DAYS_BIRTH | DAYS_EMPLOYED | DAYS_REGISTRATION | DAYS_ID_PUBLISH | OWN_CAR_AGE | FLAG_MOBIL | FLAG_EMP_PHONE | FLAG_WORK_PHONE | FLAG_CONT_MOBILE | FLAG_PHONE | FLAG_EMAIL | OCCUPATION_TYPE | CNT_FAM_MEMBERS | REGION_RATING_CLIENT | REGION_RATING_CLIENT_W_CITY | WEEKDAY_APPR_PROCESS_START | HOUR_APPR_PROCESS_START | REG_REGION_NOT_LIVE_REGION | REG_REGION_NOT_WORK_REGION | LIVE_REGION_NOT_WORK_REGION | REG_CITY_NOT_LIVE_CITY | REG_CITY_NOT_WORK_CITY | LIVE_CITY_NOT_WORK_CITY | ORGANIZATION_TYPE | EXT_SOURCE_1 | EXT_SOURCE_2 | EXT_SOURCE_3 | APARTMENTS_AVG | BASEMENTAREA_AVG | YEARS_BEGINEXPLUATATION_AVG | YEARS_BUILD_AVG | COMMONAREA_AVG | ... | APARTMENTS_MEDI | BASEMENTAREA_MEDI | YEARS_BEGINEXPLUATATION_MEDI | YEARS_BUILD_MEDI | COMMONAREA_MEDI | ELEVATORS_MEDI | ENTRANCES_MEDI | FLOORSMAX_MEDI | FLOORSMIN_MEDI | LANDAREA_MEDI | LIVINGAPARTMENTS_MEDI | LIVINGAREA_MEDI | NONLIVINGAPARTMENTS_MEDI | NONLIVINGAREA_MEDI | FONDKAPREMONT_MODE | HOUSETYPE_MODE | TOTALAREA_MODE | WALLSMATERIAL_MODE | EMERGENCYSTATE_MODE | OBS_30_CNT_SOCIAL_CIRCLE | DEF_30_CNT_SOCIAL_CIRCLE | OBS_60_CNT_SOCIAL_CIRCLE | DEF_60_CNT_SOCIAL_CIRCLE | DAYS_LAST_PHONE_CHANGE | FLAG_DOCUMENT_2 | FLAG_DOCUMENT_3 | FLAG_DOCUMENT_4 | FLAG_DOCUMENT_5 | FLAG_DOCUMENT_6 | FLAG_DOCUMENT_7 | FLAG_DOCUMENT_8 | FLAG_DOCUMENT_9 | FLAG_DOCUMENT_10 | FLAG_DOCUMENT_11 | FLAG_DOCUMENT_12 | FLAG_DOCUMENT_13 | FLAG_DOCUMENT_14 | FLAG_DOCUMENT_15 | FLAG_DOCUMENT_16 | FLAG_DOCUMENT_17 | FLAG_DOCUMENT_18 | FLAG_DOCUMENT_19 | FLAG_DOCUMENT_20 | FLAG_DOCUMENT_21 | AMT_REQ_CREDIT_BUREAU_HOUR | AMT_REQ_CREDIT_BUREAU_DAY | AMT_REQ_CREDIT_BUREAU_WEEK | AMT_REQ_CREDIT_BUREAU_MON | AMT_REQ_CREDIT_BUREAU_QRT | AMT_REQ_CREDIT_BUREAU_YEAR | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 0 | 100002 | 1.0 | Cash loans | M | N | Y | 0 | 202500.0 | 406597.5 | 24700.5 | 351000.0 | Unaccompanied | Working | Secondary / secondary special | Single / not married | House / apartment | 0.018801 | -9461 | -637 | -3648.0 | -2120 | NaN | 1 | 1 | 0 | 1 | 1 | 0 | Laborers | 1.0 | 2 | 2 | WEDNESDAY | 10 | 0 | 0 | 0 | 0 | 0 | 0 | Business Entity Type 3 | 0.083037 | 0.262949 | 0.139376 | 0.0247 | 0.0369 | 0.9722 | 0.6192 | 0.0143 | ... | 0.0250 | 0.0369 | 0.9722 | 0.6243 | 0.0144 | 0.00 | 0.0690 | 0.0833 | 0.1250 | 0.0375 | 0.0205 | 0.0193 | 0.0000 | 0.00 | reg oper account | block of flats | 0.0149 | Stone, brick | No | 2.0 | 2.0 | 2.0 | 2.0 | -1134.0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
| 1 | 1 | 100003 | 0.0 | Cash loans | F | N | N | 0 | 270000.0 | 1293502.5 | 35698.5 | 1129500.0 | Family | State servant | Higher education | Married | House / apartment | 0.003541 | -16765 | -1188 | -1186.0 | -291 | NaN | 1 | 1 | 0 | 1 | 1 | 0 | Core staff | 2.0 | 1 | 1 | MONDAY | 11 | 0 | 0 | 0 | 0 | 0 | 0 | School | 0.311267 | 0.622246 | NaN | 0.0959 | 0.0529 | 0.9851 | 0.7960 | 0.0605 | ... | 0.0968 | 0.0529 | 0.9851 | 0.7987 | 0.0608 | 0.08 | 0.0345 | 0.2917 | 0.3333 | 0.0132 | 0.0787 | 0.0558 | 0.0039 | 0.01 | reg oper account | block of flats | 0.0714 | Block | No | 1.0 | 0.0 | 1.0 | 0.0 | -828.0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 2 | 2 | 100004 | 0.0 | Revolving loans | M | Y | Y | 0 | 67500.0 | 135000.0 | 6750.0 | 135000.0 | Unaccompanied | Working | Secondary / secondary special | Single / not married | House / apartment | 0.010032 | -19046 | -225 | -4260.0 | -2531 | 26.0 | 1 | 1 | 1 | 1 | 1 | 0 | Laborers | 1.0 | 2 | 2 | MONDAY | 9 | 0 | 0 | 0 | 0 | 0 | 0 | Government | NaN | 0.555912 | 0.729567 | NaN | NaN | NaN | NaN | NaN | ... | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | 0.0 | 0.0 | 0.0 | 0.0 | -815.0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 3 | 3 | 100006 | 0.0 | Cash loans | F | N | Y | 0 | 135000.0 | 312682.5 | 29686.5 | 297000.0 | Unaccompanied | Working | Secondary / secondary special | Civil marriage | House / apartment | 0.008019 | -19005 | -3039 | -9833.0 | -2437 | NaN | 1 | 1 | 0 | 1 | 0 | 0 | Laborers | 2.0 | 2 | 2 | WEDNESDAY | 17 | 0 | 0 | 0 | 0 | 0 | 0 | Business Entity Type 3 | NaN | 0.650442 | NaN | NaN | NaN | NaN | NaN | NaN | ... | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | 2.0 | 0.0 | 2.0 | 0.0 | -617.0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | NaN | NaN | NaN | NaN | NaN | NaN |
| 4 | 4 | 100007 | 0.0 | Cash loans | M | N | Y | 0 | 121500.0 | 513000.0 | 21865.5 | 513000.0 | Unaccompanied | Working | Secondary / secondary special | Single / not married | House / apartment | 0.028663 | -19932 | -3038 | -4311.0 | -3458 | NaN | 1 | 1 | 0 | 1 | 0 | 0 | Core staff | 1.0 | 2 | 2 | THURSDAY | 11 | 0 | 0 | 0 | 0 | 1 | 1 | Religion | NaN | 0.322738 | NaN | NaN | NaN | NaN | NaN | NaN | ... | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | 0.0 | 0.0 | 0.0 | 0.0 | -1106.0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
5 rows × 123 columns
# 3.6 This gender is rare. So such
# rows can be dropped
df[df['CODE_GENDER'] == 'XNA']
| index | SK_ID_CURR | TARGET | NAME_CONTRACT_TYPE | CODE_GENDER | FLAG_OWN_CAR | FLAG_OWN_REALTY | CNT_CHILDREN | AMT_INCOME_TOTAL | AMT_CREDIT | AMT_ANNUITY | AMT_GOODS_PRICE | NAME_TYPE_SUITE | NAME_INCOME_TYPE | NAME_EDUCATION_TYPE | NAME_FAMILY_STATUS | NAME_HOUSING_TYPE | REGION_POPULATION_RELATIVE | DAYS_BIRTH | DAYS_EMPLOYED | DAYS_REGISTRATION | DAYS_ID_PUBLISH | OWN_CAR_AGE | FLAG_MOBIL | FLAG_EMP_PHONE | FLAG_WORK_PHONE | FLAG_CONT_MOBILE | FLAG_PHONE | FLAG_EMAIL | OCCUPATION_TYPE | CNT_FAM_MEMBERS | REGION_RATING_CLIENT | REGION_RATING_CLIENT_W_CITY | WEEKDAY_APPR_PROCESS_START | HOUR_APPR_PROCESS_START | REG_REGION_NOT_LIVE_REGION | REG_REGION_NOT_WORK_REGION | LIVE_REGION_NOT_WORK_REGION | REG_CITY_NOT_LIVE_CITY | REG_CITY_NOT_WORK_CITY | LIVE_CITY_NOT_WORK_CITY | ORGANIZATION_TYPE | EXT_SOURCE_1 | EXT_SOURCE_2 | EXT_SOURCE_3 | APARTMENTS_AVG | BASEMENTAREA_AVG | YEARS_BEGINEXPLUATATION_AVG | YEARS_BUILD_AVG | COMMONAREA_AVG | ... | APARTMENTS_MEDI | BASEMENTAREA_MEDI | YEARS_BEGINEXPLUATATION_MEDI | YEARS_BUILD_MEDI | COMMONAREA_MEDI | ELEVATORS_MEDI | ENTRANCES_MEDI | FLOORSMAX_MEDI | FLOORSMIN_MEDI | LANDAREA_MEDI | LIVINGAPARTMENTS_MEDI | LIVINGAREA_MEDI | NONLIVINGAPARTMENTS_MEDI | NONLIVINGAREA_MEDI | FONDKAPREMONT_MODE | HOUSETYPE_MODE | TOTALAREA_MODE | WALLSMATERIAL_MODE | EMERGENCYSTATE_MODE | OBS_30_CNT_SOCIAL_CIRCLE | DEF_30_CNT_SOCIAL_CIRCLE | OBS_60_CNT_SOCIAL_CIRCLE | DEF_60_CNT_SOCIAL_CIRCLE | DAYS_LAST_PHONE_CHANGE | FLAG_DOCUMENT_2 | FLAG_DOCUMENT_3 | FLAG_DOCUMENT_4 | FLAG_DOCUMENT_5 | FLAG_DOCUMENT_6 | FLAG_DOCUMENT_7 | FLAG_DOCUMENT_8 | FLAG_DOCUMENT_9 | FLAG_DOCUMENT_10 | FLAG_DOCUMENT_11 | FLAG_DOCUMENT_12 | FLAG_DOCUMENT_13 | FLAG_DOCUMENT_14 | FLAG_DOCUMENT_15 | FLAG_DOCUMENT_16 | FLAG_DOCUMENT_17 | FLAG_DOCUMENT_18 | FLAG_DOCUMENT_19 | FLAG_DOCUMENT_20 | FLAG_DOCUMENT_21 | AMT_REQ_CREDIT_BUREAU_HOUR | AMT_REQ_CREDIT_BUREAU_DAY | AMT_REQ_CREDIT_BUREAU_WEEK | AMT_REQ_CREDIT_BUREAU_MON | AMT_REQ_CREDIT_BUREAU_QRT | AMT_REQ_CREDIT_BUREAU_YEAR | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 35657 | 35657 | 141289 | 0.0 | Revolving loans | XNA | Y | Y | 0 | 207000.0 | 382500.0 | 19125.0 | 337500.0 | Unaccompanied | Working | Secondary / secondary special | Married | Municipal apartment | 0.020713 | -20232 | -10044 | -10024.0 | -3537 | 13.0 | 1 | 1 | 1 | 1 | 1 | 0 | NaN | 2.0 | 3 | 3 | TUESDAY | 10 | 0 | 0 | 0 | 0 | 0 | 0 | Business Entity Type 1 | NaN | 0.295998 | 0.461482 | NaN | NaN | NaN | NaN | NaN | ... | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | 0.0 | 0.0 | 0.0 | 0.0 | -286.0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
| 38566 | 38566 | 144669 | 0.0 | Revolving loans | XNA | N | Y | 2 | 157500.0 | 270000.0 | 13500.0 | 225000.0 | Family | Working | Secondary / secondary special | Married | House / apartment | 0.026392 | -13717 | -2797 | -2241.0 | -4659 | NaN | 1 | 1 | 1 | 1 | 1 | 0 | Low-skill Laborers | 4.0 | 2 | 2 | FRIDAY | 16 | 0 | 0 | 0 | 0 | 0 | 0 | Industry: type 3 | NaN | 0.709205 | 0.310818 | 0.0165 | 0.0368 | 0.9811 | NaN | NaN | ... | 0.0167 | 0.0368 | 0.9811 | NaN | NaN | 0.0 | 0.0690 | 0.0417 | NaN | 0.0237 | NaN | 0.0117 | NaN | 0.000 | NaN | block of flats | 0.0090 | Stone, brick | No | 0.0 | 0.0 | 0.0 | 0.0 | -493.0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 | 0.0 | 0.0 | 3.0 | 0.0 | 4.0 |
| 83382 | 83382 | 196708 | 0.0 | Revolving loans | XNA | N | Y | 1 | 135000.0 | 405000.0 | 20250.0 | 225000.0 | Unaccompanied | Working | Higher education | Married | House / apartment | 0.035792 | -10647 | -1228 | -183.0 | -1671 | NaN | 1 | 1 | 1 | 1 | 1 | 0 | Core staff | 3.0 | 2 | 2 | THURSDAY | 15 | 0 | 0 | 0 | 0 | 0 | 0 | Kindergarten | 0.404970 | 0.659185 | 0.076984 | 0.0773 | 0.1353 | 0.9921 | 0.8912 | NaN | ... | 0.0781 | 0.1353 | 0.9921 | 0.8927 | NaN | 0.0 | 0.1379 | 0.1667 | 0.0417 | NaN | 0.0641 | 0.0632 | 0.0000 | NaN | reg oper spec account | block of flats | 0.0769 | Panel | No | 7.0 | 1.0 | 7.0 | 1.0 | -851.0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 3.0 |
| 189640 | 189640 | 319880 | 0.0 | Revolving loans | XNA | Y | Y | 0 | 247500.0 | 540000.0 | 27000.0 | 900000.0 | Unaccompanied | Commercial associate | Incomplete higher | Civil marriage | House / apartment | 0.035792 | -9649 | -2293 | -4099.0 | -2326 | 8.0 | 1 | 1 | 1 | 1 | 1 | 0 | NaN | 2.0 | 2 | 2 | FRIDAY | 15 | 0 | 0 | 0 | 0 | 0 | 0 | Medicine | 0.653035 | 0.658620 | 0.360613 | 0.0278 | 0.0555 | 0.9712 | 0.6056 | 0.0053 | ... | 0.0281 | 0.0555 | 0.9712 | 0.6109 | 0.0054 | 0.0 | 0.0690 | 0.0833 | 0.1250 | 0.0266 | 0.0222 | 0.0298 | 0.0039 | 0.007 | reg oper account | block of flats | 0.0245 | Stone, brick | No | 10.0 | 4.0 | 10.0 | 4.0 | -1681.0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 6.0 |
4 rows × 123 columns
# 3.7 Optional: Remove 4 applications with XNA CODE_GENDER (train set)
df = df[df['CODE_GENDER'] != 'XNA']
# 3.8 Categorical features with Binary encode (0 or 1; two categories)
for bin_feature in ['CODE_GENDER', 'FLAG_OWN_CAR', 'FLAG_OWN_REALTY']:
df[bin_feature], uniques = pd.factorize(df[bin_feature])
# 3.8.1
df.head()
uniques
| index | SK_ID_CURR | TARGET | NAME_CONTRACT_TYPE | CODE_GENDER | FLAG_OWN_CAR | FLAG_OWN_REALTY | CNT_CHILDREN | AMT_INCOME_TOTAL | AMT_CREDIT | AMT_ANNUITY | AMT_GOODS_PRICE | NAME_TYPE_SUITE | NAME_INCOME_TYPE | NAME_EDUCATION_TYPE | NAME_FAMILY_STATUS | NAME_HOUSING_TYPE | REGION_POPULATION_RELATIVE | DAYS_BIRTH | DAYS_EMPLOYED | DAYS_REGISTRATION | DAYS_ID_PUBLISH | OWN_CAR_AGE | FLAG_MOBIL | FLAG_EMP_PHONE | FLAG_WORK_PHONE | FLAG_CONT_MOBILE | FLAG_PHONE | FLAG_EMAIL | OCCUPATION_TYPE | CNT_FAM_MEMBERS | REGION_RATING_CLIENT | REGION_RATING_CLIENT_W_CITY | WEEKDAY_APPR_PROCESS_START | HOUR_APPR_PROCESS_START | REG_REGION_NOT_LIVE_REGION | REG_REGION_NOT_WORK_REGION | LIVE_REGION_NOT_WORK_REGION | REG_CITY_NOT_LIVE_CITY | REG_CITY_NOT_WORK_CITY | LIVE_CITY_NOT_WORK_CITY | ORGANIZATION_TYPE | EXT_SOURCE_1 | EXT_SOURCE_2 | EXT_SOURCE_3 | APARTMENTS_AVG | BASEMENTAREA_AVG | YEARS_BEGINEXPLUATATION_AVG | YEARS_BUILD_AVG | COMMONAREA_AVG | ... | APARTMENTS_MEDI | BASEMENTAREA_MEDI | YEARS_BEGINEXPLUATATION_MEDI | YEARS_BUILD_MEDI | COMMONAREA_MEDI | ELEVATORS_MEDI | ENTRANCES_MEDI | FLOORSMAX_MEDI | FLOORSMIN_MEDI | LANDAREA_MEDI | LIVINGAPARTMENTS_MEDI | LIVINGAREA_MEDI | NONLIVINGAPARTMENTS_MEDI | NONLIVINGAREA_MEDI | FONDKAPREMONT_MODE | HOUSETYPE_MODE | TOTALAREA_MODE | WALLSMATERIAL_MODE | EMERGENCYSTATE_MODE | OBS_30_CNT_SOCIAL_CIRCLE | DEF_30_CNT_SOCIAL_CIRCLE | OBS_60_CNT_SOCIAL_CIRCLE | DEF_60_CNT_SOCIAL_CIRCLE | DAYS_LAST_PHONE_CHANGE | FLAG_DOCUMENT_2 | FLAG_DOCUMENT_3 | FLAG_DOCUMENT_4 | FLAG_DOCUMENT_5 | FLAG_DOCUMENT_6 | FLAG_DOCUMENT_7 | FLAG_DOCUMENT_8 | FLAG_DOCUMENT_9 | FLAG_DOCUMENT_10 | FLAG_DOCUMENT_11 | FLAG_DOCUMENT_12 | FLAG_DOCUMENT_13 | FLAG_DOCUMENT_14 | FLAG_DOCUMENT_15 | FLAG_DOCUMENT_16 | FLAG_DOCUMENT_17 | FLAG_DOCUMENT_18 | FLAG_DOCUMENT_19 | FLAG_DOCUMENT_20 | FLAG_DOCUMENT_21 | AMT_REQ_CREDIT_BUREAU_HOUR | AMT_REQ_CREDIT_BUREAU_DAY | AMT_REQ_CREDIT_BUREAU_WEEK | AMT_REQ_CREDIT_BUREAU_MON | AMT_REQ_CREDIT_BUREAU_QRT | AMT_REQ_CREDIT_BUREAU_YEAR | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 0 | 100002 | 1.0 | Cash loans | 0 | 0 | 0 | 0 | 202500.0 | 406597.5 | 24700.5 | 351000.0 | Unaccompanied | Working | Secondary / secondary special | Single / not married | House / apartment | 0.018801 | -9461 | -637 | -3648.0 | -2120 | NaN | 1 | 1 | 0 | 1 | 1 | 0 | Laborers | 1.0 | 2 | 2 | WEDNESDAY | 10 | 0 | 0 | 0 | 0 | 0 | 0 | Business Entity Type 3 | 0.083037 | 0.262949 | 0.139376 | 0.0247 | 0.0369 | 0.9722 | 0.6192 | 0.0143 | ... | 0.0250 | 0.0369 | 0.9722 | 0.6243 | 0.0144 | 0.00 | 0.0690 | 0.0833 | 0.1250 | 0.0375 | 0.0205 | 0.0193 | 0.0000 | 0.00 | reg oper account | block of flats | 0.0149 | Stone, brick | No | 2.0 | 2.0 | 2.0 | 2.0 | -1134.0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
| 1 | 1 | 100003 | 0.0 | Cash loans | 1 | 0 | 1 | 0 | 270000.0 | 1293502.5 | 35698.5 | 1129500.0 | Family | State servant | Higher education | Married | House / apartment | 0.003541 | -16765 | -1188 | -1186.0 | -291 | NaN | 1 | 1 | 0 | 1 | 1 | 0 | Core staff | 2.0 | 1 | 1 | MONDAY | 11 | 0 | 0 | 0 | 0 | 0 | 0 | School | 0.311267 | 0.622246 | NaN | 0.0959 | 0.0529 | 0.9851 | 0.7960 | 0.0605 | ... | 0.0968 | 0.0529 | 0.9851 | 0.7987 | 0.0608 | 0.08 | 0.0345 | 0.2917 | 0.3333 | 0.0132 | 0.0787 | 0.0558 | 0.0039 | 0.01 | reg oper account | block of flats | 0.0714 | Block | No | 1.0 | 0.0 | 1.0 | 0.0 | -828.0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 2 | 2 | 100004 | 0.0 | Revolving loans | 0 | 1 | 0 | 0 | 67500.0 | 135000.0 | 6750.0 | 135000.0 | Unaccompanied | Working | Secondary / secondary special | Single / not married | House / apartment | 0.010032 | -19046 | -225 | -4260.0 | -2531 | 26.0 | 1 | 1 | 1 | 1 | 1 | 0 | Laborers | 1.0 | 2 | 2 | MONDAY | 9 | 0 | 0 | 0 | 0 | 0 | 0 | Government | NaN | 0.555912 | 0.729567 | NaN | NaN | NaN | NaN | NaN | ... | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | 0.0 | 0.0 | 0.0 | 0.0 | -815.0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 3 | 3 | 100006 | 0.0 | Cash loans | 1 | 0 | 0 | 0 | 135000.0 | 312682.5 | 29686.5 | 297000.0 | Unaccompanied | Working | Secondary / secondary special | Civil marriage | House / apartment | 0.008019 | -19005 | -3039 | -9833.0 | -2437 | NaN | 1 | 1 | 0 | 1 | 0 | 0 | Laborers | 2.0 | 2 | 2 | WEDNESDAY | 17 | 0 | 0 | 0 | 0 | 0 | 0 | Business Entity Type 3 | NaN | 0.650442 | NaN | NaN | NaN | NaN | NaN | NaN | ... | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | 2.0 | 0.0 | 2.0 | 0.0 | -617.0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | NaN | NaN | NaN | NaN | NaN | NaN |
| 4 | 4 | 100007 | 0.0 | Cash loans | 0 | 0 | 0 | 0 | 121500.0 | 513000.0 | 21865.5 | 513000.0 | Unaccompanied | Working | Secondary / secondary special | Single / not married | House / apartment | 0.028663 | -19932 | -3038 | -4311.0 | -3458 | NaN | 1 | 1 | 0 | 1 | 0 | 0 | Core staff | 1.0 | 2 | 2 | THURSDAY | 11 | 0 | 0 | 0 | 0 | 1 | 1 | Religion | NaN | 0.322738 | NaN | NaN | NaN | NaN | NaN | NaN | ... | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | 0.0 | 0.0 | 0.0 | 0.0 | -1106.0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
5 rows × 123 columns
Index(['Y', 'N'], dtype='object')
# 3.8.2
df.dtypes
df.dtypes.value_counts()
df.dtypes['CODE_GENDER']
df.dtypes['FLAG_OWN_CAR']
df.dtypes['FLAG_OWN_REALTY']
index int64
SK_ID_CURR uint32
TARGET float64
NAME_CONTRACT_TYPE object
CODE_GENDER int64
...
AMT_REQ_CREDIT_BUREAU_DAY float64
AMT_REQ_CREDIT_BUREAU_WEEK float64
AMT_REQ_CREDIT_BUREAU_MON float64
AMT_REQ_CREDIT_BUREAU_QRT float64
AMT_REQ_CREDIT_BUREAU_YEAR float64
Length: 123, dtype: object
float64 62
uint8 35
object 13
int64 4
float32 4
int16 3
int32 1
uint32 1
dtype: int64
dtype('int64')
dtype('int64')
dtype('int64')
# 4.0 Categorical features with One-Hot encode
df, cat_cols = one_hot_encoder(df, nan_as_category)
# 4.1
len(cat_cols) # 146
cat_cols
146
['NAME_CONTRACT_TYPE_Cash loans',
'NAME_CONTRACT_TYPE_Revolving loans',
'NAME_CONTRACT_TYPE_nan',
'NAME_TYPE_SUITE_Children',
'NAME_TYPE_SUITE_Family',
'NAME_TYPE_SUITE_Group of people',
'NAME_TYPE_SUITE_Other_A',
'NAME_TYPE_SUITE_Other_B',
'NAME_TYPE_SUITE_Spouse, partner',
'NAME_TYPE_SUITE_Unaccompanied',
'NAME_TYPE_SUITE_nan',
'NAME_INCOME_TYPE_Businessman',
'NAME_INCOME_TYPE_Commercial associate',
'NAME_INCOME_TYPE_Maternity leave',
'NAME_INCOME_TYPE_Pensioner',
'NAME_INCOME_TYPE_State servant',
'NAME_INCOME_TYPE_Student',
'NAME_INCOME_TYPE_Unemployed',
'NAME_INCOME_TYPE_Working',
'NAME_INCOME_TYPE_nan',
'NAME_EDUCATION_TYPE_Academic degree',
'NAME_EDUCATION_TYPE_Higher education',
'NAME_EDUCATION_TYPE_Incomplete higher',
'NAME_EDUCATION_TYPE_Lower secondary',
'NAME_EDUCATION_TYPE_Secondary / secondary special',
'NAME_EDUCATION_TYPE_nan',
'NAME_FAMILY_STATUS_Civil marriage',
'NAME_FAMILY_STATUS_Married',
'NAME_FAMILY_STATUS_Separated',
'NAME_FAMILY_STATUS_Single / not married',
'NAME_FAMILY_STATUS_Unknown',
'NAME_FAMILY_STATUS_Widow',
'NAME_FAMILY_STATUS_nan',
'NAME_HOUSING_TYPE_Co-op apartment',
'NAME_HOUSING_TYPE_House / apartment',
'NAME_HOUSING_TYPE_Municipal apartment',
'NAME_HOUSING_TYPE_Office apartment',
'NAME_HOUSING_TYPE_Rented apartment',
'NAME_HOUSING_TYPE_With parents',
'NAME_HOUSING_TYPE_nan',
'OCCUPATION_TYPE_Accountants',
'OCCUPATION_TYPE_Cleaning staff',
'OCCUPATION_TYPE_Cooking staff',
'OCCUPATION_TYPE_Core staff',
'OCCUPATION_TYPE_Drivers',
'OCCUPATION_TYPE_HR staff',
'OCCUPATION_TYPE_High skill tech staff',
'OCCUPATION_TYPE_IT staff',
'OCCUPATION_TYPE_Laborers',
'OCCUPATION_TYPE_Low-skill Laborers',
'OCCUPATION_TYPE_Managers',
'OCCUPATION_TYPE_Medicine staff',
'OCCUPATION_TYPE_Private service staff',
'OCCUPATION_TYPE_Realty agents',
'OCCUPATION_TYPE_Sales staff',
'OCCUPATION_TYPE_Secretaries',
'OCCUPATION_TYPE_Security staff',
'OCCUPATION_TYPE_Waiters/barmen staff',
'OCCUPATION_TYPE_nan',
'WEEKDAY_APPR_PROCESS_START_FRIDAY',
'WEEKDAY_APPR_PROCESS_START_MONDAY',
'WEEKDAY_APPR_PROCESS_START_SATURDAY',
'WEEKDAY_APPR_PROCESS_START_SUNDAY',
'WEEKDAY_APPR_PROCESS_START_THURSDAY',
'WEEKDAY_APPR_PROCESS_START_TUESDAY',
'WEEKDAY_APPR_PROCESS_START_WEDNESDAY',
'WEEKDAY_APPR_PROCESS_START_nan',
'ORGANIZATION_TYPE_Advertising',
'ORGANIZATION_TYPE_Agriculture',
'ORGANIZATION_TYPE_Bank',
'ORGANIZATION_TYPE_Business Entity Type 1',
'ORGANIZATION_TYPE_Business Entity Type 2',
'ORGANIZATION_TYPE_Business Entity Type 3',
'ORGANIZATION_TYPE_Cleaning',
'ORGANIZATION_TYPE_Construction',
'ORGANIZATION_TYPE_Culture',
'ORGANIZATION_TYPE_Electricity',
'ORGANIZATION_TYPE_Emergency',
'ORGANIZATION_TYPE_Government',
'ORGANIZATION_TYPE_Hotel',
'ORGANIZATION_TYPE_Housing',
'ORGANIZATION_TYPE_Industry: type 1',
'ORGANIZATION_TYPE_Industry: type 10',
'ORGANIZATION_TYPE_Industry: type 11',
'ORGANIZATION_TYPE_Industry: type 12',
'ORGANIZATION_TYPE_Industry: type 13',
'ORGANIZATION_TYPE_Industry: type 2',
'ORGANIZATION_TYPE_Industry: type 3',
'ORGANIZATION_TYPE_Industry: type 4',
'ORGANIZATION_TYPE_Industry: type 5',
'ORGANIZATION_TYPE_Industry: type 6',
'ORGANIZATION_TYPE_Industry: type 7',
'ORGANIZATION_TYPE_Industry: type 8',
'ORGANIZATION_TYPE_Industry: type 9',
'ORGANIZATION_TYPE_Insurance',
'ORGANIZATION_TYPE_Kindergarten',
'ORGANIZATION_TYPE_Legal Services',
'ORGANIZATION_TYPE_Medicine',
'ORGANIZATION_TYPE_Military',
'ORGANIZATION_TYPE_Mobile',
'ORGANIZATION_TYPE_Other',
'ORGANIZATION_TYPE_Police',
'ORGANIZATION_TYPE_Postal',
'ORGANIZATION_TYPE_Realtor',
'ORGANIZATION_TYPE_Religion',
'ORGANIZATION_TYPE_Restaurant',
'ORGANIZATION_TYPE_School',
'ORGANIZATION_TYPE_Security',
'ORGANIZATION_TYPE_Security Ministries',
'ORGANIZATION_TYPE_Self-employed',
'ORGANIZATION_TYPE_Services',
'ORGANIZATION_TYPE_Telecom',
'ORGANIZATION_TYPE_Trade: type 1',
'ORGANIZATION_TYPE_Trade: type 2',
'ORGANIZATION_TYPE_Trade: type 3',
'ORGANIZATION_TYPE_Trade: type 4',
'ORGANIZATION_TYPE_Trade: type 5',
'ORGANIZATION_TYPE_Trade: type 6',
'ORGANIZATION_TYPE_Trade: type 7',
'ORGANIZATION_TYPE_Transport: type 1',
'ORGANIZATION_TYPE_Transport: type 2',
'ORGANIZATION_TYPE_Transport: type 3',
'ORGANIZATION_TYPE_Transport: type 4',
'ORGANIZATION_TYPE_University',
'ORGANIZATION_TYPE_XNA',
'ORGANIZATION_TYPE_nan',
'FONDKAPREMONT_MODE_not specified',
'FONDKAPREMONT_MODE_org spec account',
'FONDKAPREMONT_MODE_reg oper account',
'FONDKAPREMONT_MODE_reg oper spec account',
'FONDKAPREMONT_MODE_nan',
'HOUSETYPE_MODE_block of flats',
'HOUSETYPE_MODE_specific housing',
'HOUSETYPE_MODE_terraced house',
'HOUSETYPE_MODE_nan',
'WALLSMATERIAL_MODE_Block',
'WALLSMATERIAL_MODE_Mixed',
'WALLSMATERIAL_MODE_Monolithic',
'WALLSMATERIAL_MODE_Others',
'WALLSMATERIAL_MODE_Panel',
'WALLSMATERIAL_MODE_Stone, brick',
'WALLSMATERIAL_MODE_Wooden',
'WALLSMATERIAL_MODE_nan',
'EMERGENCYSTATE_MODE_No',
'EMERGENCYSTATE_MODE_Yes',
'EMERGENCYSTATE_MODE_nan']
# 4.2 NaN values for DAYS_EMPLOYED: 365.243 -> nan
df['DAYS_EMPLOYED'].replace(365243, np.nan, inplace= True)
# 4.3 Some simple new features (percentages)
df['DAYS_EMPLOYED_PERC'] = df['DAYS_EMPLOYED'] / df['DAYS_BIRTH']
df['INCOME_CREDIT_PERC'] = df['AMT_INCOME_TOTAL'] / df['AMT_CREDIT']
df['INCOME_PER_PERSON'] = df['AMT_INCOME_TOTAL'] / df['CNT_FAM_MEMBERS']
df['ANNUITY_INCOME_PERC'] = df['AMT_ANNUITY'] / df['AMT_INCOME_TOTAL']
df['PAYMENT_RATE'] = df['AMT_ANNUITY'] / df['AMT_CREDIT']
# 5.0 Save the results for subsequent use:
df.to_csv("processed_df.csv.zip", compression = "zip")
#################