Created by <Academia de Código_>
Collection of web components that work together to provide a specific functionality on the Web
In the JEE specification, a web component is either a Servlet, JavaServer Faces or a Java Server Page (JSP) page Web applications are modeled after the request-response stateless nature of the HTTP protocolCore classes in web applications that either perform the work of responding to requests or delegate that work to some other part of the application
The web application and its constituent components are managed and executed inside the web container, also called a servlet container.
JSP is an extension of the servlet technology allowing java programming code to be embedded in HTML pages
The Java code is executed on the server and compiled with the JSP to produce a plain HTML page
JSP files are first compiled into Servlets before they can be used
The javax.servlet.Servlet interface defines lifecycle methods to initialize a servlet, to service requests and to remove a servlet from the server
The interface also provides the getServletConfig method, which the servlet can use to get any startup information and the getServletInfo method, which allows the servlet to return basic information about itselfThe Servlet life-cycle methods are invoked by the container at appropriate instants in a servlet’s life:
Can be used to access application parameters (request, initialization and context), set globally accessible attributes and to obtain container information
javax.servlet.http.HttpServlet extends GenericServlet and implements the service method to accept only HTTP requests, providing empty implementations for methods corresponding to each HTTP verb type
In a web application the request flow can be redirected to a completely different URL
The redirect is done by the browser and can be triggered by calling sendRedirect() on the response object.
public class LoginServlet extends HttpServlet {
@Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp) {
resp.sendRedirect("http://javabank.com/login");
}
}
Requests can be dispatched to other application components such as Servlets or JSP pages
A request dispatch is different from a redirect, the work is done on the server side and not by the browser
public class LoginServlet extends HttpServlet {
@Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp) {
RequestDispatcher dispatcher = getServletContext().getRequestDispatcher("customer.jsp");
dispatcher.forward(req, resp);
}
}
Filters are application components organized in a chain that can intercept, inspect and modify requests and responses
Filters implement the javax.servlet.Filter interface, are configured on the deployment descriptor and are typically used for authentication, caching, encryption, logging, auditing, etc.
Web Applications follow a directory structure convention, which contains the deployment descriptor, classes and other application resources
A WAR file is usually used to distribute a collection of JavaServer Pages, Java Servlets, Java classes, XML files, tag libraries, static Web pages (HTML and related files) and other resources that together constitute a Web application
In essence, a WAR file is really a JAR file with some
additional web specific metadata
At the heart of all web applications is a deployment descriptor that is an XML file named web.xml and contains configuration information for the entire web application.
Java Based Web Application Container used to deploy Java applications that are built using the Servlet and JSP APIs
Provides goals to deploy and manipulate WAR projects within the Apache Tomcat servlet container
Once a WAR project has been deployed on Tomcat, it is possible to:
<plugin>
<groupId>org.apache.tomcat.maven</groupId>
<artifactId>tomcat7-maven-plugin</artifactId>
<version>2.2</version>
<configuration>
<url>http://localhost:8080/manager/text</url>

<server>tomcat</server>
<path>/hello</path>
</configuration>
</plugin>
The View can be presented by forwarding the request to the JSP using a RequestDispatcher
The RequestDispatcher object receives requests from the client and sends them to any resource on the server, such as a servlet, HTML file or JSP file
// Create request dispatcher wrappers around the views
RequestDispatcher page1Dispatcher = getServletContext().getRequestDispatcher("page1.jsp");
RequestDispatcher page2Dispatcher = getServletContext().getRequestDispatcher("page2.jsp");
// include the page1 processing by calling the appropriate jsp page code
page1Dispatcher.include(request, response);
// transfer the control to page2 jsp
page2Dispatcher.forward(request, response);
Before handling the processing of the request to the view, it needs to be populated with model data
User user = new User();
user.setName("Catarina Campino");
user.setEmail("catarina.campino@academiadecodigo.org");
// Use global Servlet context to pass user object
getServletContext().setAttribute("user", user);
// Use session context to pass the user object
request.getSession().setAttribute("user", user);
// Use request context to pass the user object
request.setAttribute("user", user);
Session state is maintained in Tomcat using a JSESSIONID cookie, which is created the first time request.getSession()
Allows dynamic access to model data stored as attributes in the request, session or application scope

Welcome ${user.name}

Email: ${user.email}

The apache maven-archetype-webapp can be utilized to create a basic web application structure
Make sure the maven compiler and tomcat plugins are installed and configured and the servlet-api dependency is added
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
<version>3.1.0</version>
<scope>provided</scope>
</dependency>
The
provided scope means the dependency will be provided at runtime by the compiler
Higher level abstraction built on top of Java Servlets and centered around the Model View Controller design pattern
Speeds up development time by providing general solutions to commonly well-known problems
Spring uses the Front Controller design pattern
Servlets are replaced with controllers
DispatcherServlet and HandlerMapping beans are provided by the framework
Spring will register controllers and their mappings by scanning your code for the @Controller and @RequestMapping annotations
// Let Spring find the controller
@Controller
public class HelloController {
// Map the URL/Verb to the method
@RequestMapping(method = RequestMethod.GET, value = "/hello")
public String sayHello(Model model) {
// Add an attribute to the request
model.addAttribute("greeting", "Hello World");
// Return the view name
return "hello";
}
}
Model data can be conveniently added to the request through The model is a Map interface defining a holder for model attributes and allowing for the complete abstraction of the view technology
The ViewResolver interface provides a mapping between view names and the actual view objects
InternalResourceViewResolver is a convenience ViewResolver implementation for using JSP pages as view templates
The view resolver surrounds the view name with a prefix and a suffix
providing the dispatcher servlet with the file path to the template
<%@ page contentType="text/html;charset=UTF-8" language="java" %>

<title>Hello Spring MVC</title>

${greeting}

Model data is accessed using the JSP Expression Language
<beans>

<mvc:default-servlet-handler/>

<mvc:annotation-driven/>
<context:component-scan base-package="org.academiadecodigo.bootcamp.controller"/>

<bean class="org.springframework.web.servlet.view.InternalResourceViewResolver">
<property name="prefix" value="/templates/"/>
<property name="suffix" value=".jsp"/>
</bean>
</beans>
The Spring application context is loaded from the DispatcherServlet, which is configured in the project deployment descriptor:
<servlet>
<servlet-name>dispatcherServlet</servlet-name>
<servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
<init-param>
<param-name>contextConfigLocation</param-name>
<param-value>WEB-INF/spring/spring-config.xml</param-value>
</init-param>
</servlet>
<servlet-mapping>
<servlet-name>dispatcherServlet</servlet-name>
<url-pattern>/</url-pattern>
</servlet-mapping>
<dependencies>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-webmvc</artifactId>
<version>4.3.11.RELEASE</version>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
<version>3.1.0</version>
<scope>provided</scope>
</dependency>
</dependencies>
Add these to maven-archetype-webapp project
Spring auto wiring enables object dependencies to be injected implicitly without the need for explicit configuration
Beans can be registered on the Spring container by using one of the available annotations:
A Spring managed bean can be injected into a property simply by using the @Autowired annotation
@Service
@Profile("jpa")
public class CustomerServiceImpl implements CustomerService {}
@Service
@Profile("memory")
public class MockCustomerService implements CustomerService {}
@Controller
@RequestMapping("/customer")
public class CustomerController {
private CustomerService customerService;
@Autowired
public void setCustomerService(CustomerService customerService) {}
}
Active profile selection can be done through servlet context parameter:
<context-param>
<param-name>spring.profiles.active</param-name>
<param-value>jpa</param-value>
</context-param>
Spring allows the usage of different view technologies, from the well established JSP technology to a variety of template engines
To render each view type, a ViewResolver bean corresponding to the template engine technology needs to be defined
Modern Java template engine which can process HTML, XML, text, JavaScript or CSS files
Supports Natural Templating, allowing the template to be correctly displayed in the browser and work as a static prototype
For simple model attributes, the th:text tag attribute can be used to display the value of model attributes:

Welcome Bruno Ferreira

For iterating over a collection, the th:each tag attribute can be used:

The
th:if attribute is used to display a section of the view
if the condition
is met
The
th:unless attribute is used to display a section of the view
if the condition
is not met

The th:switch and th:case attributes are used to display content conditionally:

<!DOCTYPE html>

<title>JavaBank</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>

Customer List

Id
First Name
Last Name
Email
Phone

<bean id="viewResolver" class="org.thymeleaf.spring4.view.ThymeleafViewResolver">
<property name="templateEngine" ref="templateEngine"/>
<property name="characterEncoding" value="UTF-8"/>
<property name="contentType" value="text/html; charset=UTF-8"/>
</bean>
The
ThymeleafViewResolver bean requires two additional beans from the Thymeleaf project to work:

<bean id="templateEngine" class="org.thymeleaf.spring4.SpringTemplateEngine">
<property name="templateResolvers">
<set>
<ref bean="templateResolver"/>
</set>
</property>
</bean>

<bean id="templateResolver" class="org.thymeleaf.spring4.templateresolver.SpringResourceTemplateResolver">
<property name="prefix" value="/templates/"/>
<property name="suffix" value=".html"/>
<property name="templateMode" value="HTML"/>
<property name="cacheable" value="false"/>
<property name="characterEncoding" value="UTF-8"/>
</bean>
The Spring Thymeleaf package needs to be added
to the project metadata:
<dependency>
<groupId>org.thymeleaf</groupId>
<artifactId>thymeleaf-spring4</artifactId>
<version>3.0.8.RELEASE</version>
</dependency>
The RequestParam annotation indicates that a method parameter should be bound to an URL request parameter
// Can serve URLs like http://www.someserver.org/someapp/hello?name=catarina
@RequestMapping(method = RequestMethod.GET, value = "/hello")
public String hello(Model model, @RequestParam("name") String name) {
model.addAttribute("message", "Hello " + name);
return "hello";
}
URL-like string, containing one or more variable names. When these variables are substituted by values, the template becomes a URL
The @PathVariable annotation can be used on a controller method argument to bind it to the value of a URI template variable:
@RequestMapping(method = RequestMethod.GET, path = "/customer/{id}")
public String getCustomer(@PathVariable Integer id) {
// User asked for a specific customer
}
The SessionAttributes annotation is used on a controller to designate which model attributes should be stored in the session before rendering the next view
@Controller
@SessionAttributes("user")
public class LoginController {
// store accessing user in the http session
}
A Browser redirect can be triggered by returning from the controller the redirect: prefix followed by the URL:
public String deleteCustomer(@PathVariable Integer id) {
customerService.delete(id);
return "redirect:/customer";
}
Command objects (or form objects, form-backing objects) allow binding of form request parameters to bean properties using setters
<form th:object="${customer}" th:action="@{/customer}" method="post">
<input type="text" th:field="*{firstName}"/>
</form>
Values for
th:field attributes must be selection expressions
Thymeleaf attributes have their values set by using
one of the following expression syntax:
Data Transfer Object is a design pattern used to transport information from a remote client to a server and vice-versa
Using DTOs instead of models to transfer data to/from Spring backed forms we separate presentation logic from persistence logic
A DTO is just a collection of properties with getters and setters and should NOT have complex associations
A converter/assembler is required on the server to transfer data between DTOs and domain/model objects
If the controller processing a form submission returns a success view, hitting the refresh browser button may trigger a new submission of the form with the same form data:
PRG is a web development design pattern
that prevents duplicate form submissions
To implement PRG the controller handling the request
needs to redirect the browser to another URL:
@Controller
@RequestMapping("/customer")
public class CustomerController {
@RequestMapping(method = RequestMethod.GET, path = "/{id}")
public String showCustomer(@PathVariable Integer id, Model model) {
model.addAttribute("customer", customerService.get(id));
return "customer/show";
}
@RequestMapping(method = RequestMethod.POST, value = "/add")
public String addCustomer(@ModelAttribute Customer customer) {
Customer savedCustomer = customerService.save(customer);
// Instead of returning a rendered view to the browser,
// a 302 redirect is sent to the browser, forcing showCustomer()
// to execute after adding a new user
return "redirect:/customer/" + savedCustomer.getId();
}
}
When redirecting a request, data stored in the original request is lost, making it impossible to the controller handling the redirected request to access the parameters contained in the original one
PRG solves multiple form submissions but prevents retrieving request parameters and attributes
Flash attributes provide a way for one request to store attributes intended for use in another
request
@Controller
@RequestMapping("customer")
public class CustomerController {
@RequestMapping(method = RequestMethod.GET, path = {"/list", "/", ""})
public String listCustomers(Model model) {
model.addAttribute("customers", customerService.list());
return "customer/list";
}
@RequestMapping(method = RequestMethod.POST, path = {"/", ""})
public String saveCustomer(Customer customer, RedirectAttributes redirectAttributes) {
Customer savedCustomer = customerService.save(customer);
redirectAttributes.addFlashAttribute("lastAction", "Added customer successfully!");
return "redirect:/customer/list";
}
}
Flash attributes are saved temporarily before the redirect and removed immediately after
JSR 303 (part of J2EE) defines a metadata model and API
for JavaBeans validation
Spring MVC is JSR 303 compatible
JSR 303 reference implementation, validation rules expressed in a standardized way using annotation-based constraints

<dependency>
<groupId>org.hibernate.validator</groupId>
<artifactId>hibernate-validator</artifactId>
<version>6.0.2.Final</version>
</dependency>
Validation rules and error messages are defined
through annotations in the DTO class
public class CustomerDto {
private Integer id;
@NotNull(message = "first name is mandatory")
@NotBlank(message = "first name is mandatory")
@Size(min=3, max=64)
private String firstName;
@NotNull(message = "first name is mandatory")
@NotBlank(message = "first name is mandatory")
@Size(min=3, max=64)
private String lastName;
@Email
private String email;
@Pattern(regexp = "^\+?[0-9]*$", message = "phone has invalid characters")
@Size(min=9, max=16)
private String phone;
// Getters and setters ...
}
@Valid annotation on the command argument
BindingResult object with validation results
@Controller
@RequestMapping("customer")
public class CustomerController {
@RequestMapping(method = RequestMethod.POST, path = {"/", ""})
public String saveCustomer(@Valid CustomerDto newCustomer, BindingResult bindingResult) {
if (bindingResult.hasErrors()) {
return "customer/add-update";
}
// save customer
}
}
The
BindingResult argument must
immediately follow the command object
The function
#fields.hasErrors() can be used
to check if a field has errors
With
#fields.errors() or
th:errors the errors
for a particular field can be displayed
First Name:
<input type="text" th:field="*{firstName}"/>

First Name field error messages

Dealing with the sunshine path in our application is not enough
How should we handle an action on a Customer that does not exist?
Code to handle application domains errors is required!
Creating dedicated exceptions for signaling business logic errors allows specialized error handling for each situation
Lets start small with a single
JavaBankException
and handle all generic business logic errors in one centralized place
How can we achieve this?
Annotating a class with
@ControllerAdvice
applies its methods
to all controllers in the application
Annotations like @ExceptionHandler in a specific method act as selectors, calling the method when the selected exception is thrown
Armed with this information, let's create a generic exception handler
Controller responsible for rendering
a generic error view on uncaught exceptions
@ControllerAdvice
public class GlobalControllerExceptionHandler {
@ResponseStatus(HttpStatus.BAD_REQUEST) // respond with an http 400
@ExceptionHandler(value = JavaBankException.class) // user did something wrong
public ModelAndView handleClientErrors(HttpServletRequest req, JavaBankException ex) {
ModelAndView model = new ModelAndView();
// add error info to the model
model.setViewName("error");
return model;
}
@ResponseStatus(HttpStatus.INTERNAL_SERVER_ERROR) // respond with an http 500
@ExceptionHandler(value = Exception.class) // something is wrong with the server
public ModelAndView handleServerErrors(HttpServletRequest req, Exception ex) {
ModelAndView model = new ModelAndView();
// add error info to the model
model.setViewName("error");
return model;
}
}
Now that we have a proper error page for uncaught exceptions,
we can add specific error handling logic to controllers
@RequestMapping(method = RequestMethod.GET, path = "/{id}/delete")
public String deleteCustomer(@PathVariable Integer id, RedirectAttributes redirectAttributes) {
try {
Customer customer = customerService.get(id);
customerService.delete(id);
// populate the model and render the appropriate view
} catch (AssociationExistsException) {
// provide a visual indication to the user that is not a good idea
// to delete the customer before closing all accounts
}
}
Concrete business logic exceptions
should extend from our generic
JavaBankException
The process of writing messages regarding a running piece of software to a centralized place
These messages have different levels to reflect their importance:
The object which performs the logging is called a Logger
Logging allows us to persist important information such as errors, client accesses, transactions and other relevant events.
Log files are mainly used for debugging, data analysis and statistics.
Stack traces are useful for knowing the exact point where an application failed, but they provide little to no information regarding the previous state of an application
Three major logging frameworks to choose from exist:
JUL cannot be recommended due to a lower performance when compared to the other options
SLF4J is a logging facade, which abstracts the various logging frameworks from the application

<dependencies>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.10.0</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.10.0</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-web</artifactId>
<version>2.10.0</version>
</dependency>
</dependencies>
Loggers created with the LogManager#getLogger() method
It's accepted as a common practice to request one logger per class. This way, our log writings are easily identifiable.
public class MySpecialClass {
// getting the class logger
// by convention, the property's name is logger
private Logger byClass = LogManager.getLogger(MySpecialClass.class);
private Logger byName = LogManager.getLogger("org.academiadecodigo.bootcamp.MySpecialClass");
public static void main(String[] args) {
// logging different levels
byClass.trace("Trace log message");
byClass.debug("Debug log message");
byClass.info("Info log message");
byClass.warn("Warn log message");
byClass.error("Error log message");
byClass.fatal("Fatal log message");
}
}
Log4j2 can be configured under various formats:
Its two main elements are Appenders and Loggers
Appenders write event data to a target location. We can choose to write this data under a given format by using layouts.
A layout allows us to choose which data elements we want to include on each of the records written by a specific appender
Objects which we have to interact with in order to perform logging.
Each logger can have multiple appenders and will delegate to one of the appenders the operation of writing to an output.
At the very least the Root logger must be configured!
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="ALL">
<Appenders>
<RollingFile name="AccessRollingFile"
fileName="${sys:catalina.home}/logs/javabank/access/javabank_access.log"
filePattern="${sys:catalina.home}/logs/javabank/access/$${date:yyyy}/$${date:MM}/javabank_access_%d{dd}.log"
append="false">
<PatternLayout>
<Pattern>%d [%t] - %p - %c - %m%n</Pattern>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy interval="1" modulate="true"/>
</Policies>
<LevelRangeFilter minLevel="INFO" maxLevel="INFO"/>
</RollingFile>
<RollingFile name="ErrorRollingFile"
fileName="${sys:catalina.home}/logs/javabank/error/javabank_error.log"
filePattern="${sys:catalina.home}/logs/javabank/error/$${date:yyyy}/$${date:MM}/javabank_error_%d{dd}.log"
append="false">
<PatternLayout>
<Pattern>%d [%t] - %p - %c - %m%n</Pattern>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy interval="1" modulate="true"/>
</Policies>
<ThresholdFilter level="WARN" onMatch="ACCEPT" onMismatch="DENY"/>
</RollingFile>
<RollingFile name="DebugRollingFile" fileName="${sys:catalina.home}/logs/javabank/debug/javabank_debug.log"
filePattern="${sys:catalina.home}/logs/javabank/debug/$${date:yyyy}/$${date:MM}/javabank_debug_%d{dd}.log"
append="false">
<PatternLayout>
<Pattern>%d [%t] - %p - %c - %m%n</Pattern>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy interval="1" modulate="true"/>
</Policies>
</RollingFile>
</Appenders>
<Loggers>
<Root level="INFO">
<AppenderRef ref="AccessRollingFile"/>
<AppenderRef ref="ErrorRollingFile"/>
</Root>
<Logger name="org.hibernate.SQL" level="DEBUG" additivity="false>
<AppenderRef ref="DebugRollingFile"/>
</Logger>
</Loggers>
</Configuration>
Popular Distributed Application Frameworks
used to implement SOA include:
Usage of the Web Platform as a connection technology for service-oriented architectures.The benefits are:
Architectural style in which systems communicate over HTTP using the same verbs that web browsers use to retrieve web pages and to send data to remote servers. Such systems are called RESTful.
| HTTP Verb | /tasks | /tasks/{id} |
|---|---|---|
| GET | 200 (OK), list of tasks. Use pagination, sorting and filtering to navigate big lists. | 200 (OK), single task. 404 (Not Found), if ID not found or invalid. |
| PUT | 404 (Not Found), unless you want to update/replace every resource in the entire collection. | 200 (OK) or 204 (No Content). 404 (Not Found), if ID not found or invalid. |
| POST | 201 (Created). Response also contains a link to the created task (/task/{id}) in its header. | 404 (Not Found). |
| DELETE | 404 (Not Found), unless you want to delete the whole collection which is not often desirable. | 200 (OK). 404 (Not Found), if ID not found or invalid. |
Services are exposed as a REST API
API can be consumed by different types of clients/devices
Provides a framework to serialize custom java objects to a json string and deserialize a json string back into a java objects.
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.8.0</version>
</dependency>
How do we use it?
import com.fasterxml.jackson.databind.ObjectMapper;
// ...
ObjectMapper mapper = new ObjectMapper();
public class MyPojo {
private String name;
private int age;
// Getters and Setters
}
MyPojo myPojo1 = mapper.readValue(new File("data.json"), MyPojo.class);
MyPojo myPojo2 = mapper.readValue(new URL("http://api.com/data.json"), MyPojo.class);
MyPojo myPojo3 = mapper.readValue("{\"name\":\"Pedro\", \"age\":23}", MyPojo.class);
public class MyPojo {
private String name;
private int age;
// Getters and Setters
}
MyPojo myPojo = new MyPojo();
myPojo.setName("Pedro");
myPojo.setAge(23);
mapper.writeValue(new File("result.json"), myPojo);
byte[] jsonBytes = mapper.writeValueAsBytes(myPojo);
String jsonString = mapper.writeValueAsString(myPojo);
@RequestBody and @ResponseBody can be used in a controller to implement smart object serialization and deserialization
They extract the logic of message conversion from the code allow multiple data formats for a single resource without code duplication
@Controller
@ResponseBody
public class HelloWorldController {
@RequestMapping(method = RequestMethod.GET, value = "api/hello")
public String helloWorld(@RequestParam(value = "name", defaultValue = "World") String str) {
return "Hello " + str;
}
}
Annotating a class with
@RestController is the same as annotating it with both
@Controller and
@ResponseBody
The ResponseEntity class represents an HTTP response, consisting of headers, body and response code
It is used to return a specific http status code from the controller
@RestController
public class HelloWorldController {
@RequestMapping(method = RequestMethod.GET, value = "api/hello")
public ResponseEntity<String>
helloWorld(@RequestParam(value = "name", defaultValue = "World") String str) {
return new ResponseEntity<String>("Hello " + str, HttpStatus.I_AM_A_TEAPOT);
}
}
Spring will use Jackson to serialize and deserialize objects automatically if it finds it in the class path
public class Message {
private String message;
// getters and setters
}
@RestController
public class HelloWorldController {
@RequestMapping(
method = RequestMethod.GET,
value = "api/hello",
produces = MediaType.APPLICATION_JSON_VALUE
)
public ResponseEntity<Message>
helloWorld(@RequestParam(value = "name", defaultValue = "World") String str) {
Message message = new Message();
message.setMessage("Hello " + str);
return new ResponseEntity<>(message, HttpStatus.OK);
}
}
Remember the @Valid annotation? We can validate the deserialized JSON object by adding @Valid to arguments
By using the BindingResult class we can access the evaluation result
@RestController
public class UserController {
@RequestMapping(
method = RequestMethod.POST,
value = "api/users",
produces = MediaType.APPLICATION_JSON_VALUE
)
public ResponseEntity addUser(@Valid @RequestBody User user, BindingResult bindingResult) {
if (bindingResult.hasErrors()) {
return new ResponseEntity<>(HttpStatus.BAD_REQUEST);
}
// ...
}
}
The
BindingResult argument must
immediately follow the object to be validated
Test with Postman / curl
| Http Method | EndPoint | Action |
|---|---|---|
| GET | /customer | read all customers, no account info |
| GET | /customer/{id} | read a specific customer |
| GET | /customer/{id}/account | read all accounts for customer |
| GET | /customer/{cid}/account/{aid} | read account for customer |
| POST | /customer | create a new customer |
| PUT | /customer/{id} | edit an existing customer |
| DELETE | /customer/{id} | delete an existing customer |
| DELETE | /customer/{cid}/account/{aid} | delete an account |
One of the most misunderstood and misimplemented concepts
Security is like onions, it comes in layers
Traditional web applications communicate through
HTTP,
which by nature is insecure
A secure approach to communication between client and server is using HTTPS ( HTTP + TLS)
This can be provided by your application container e.g: Tomcat
This an extensive topic and we'll only cover mostly 2 topics:
The most common authentication mechanism is the client providing the server with a combination of a user and password
Examples of other mechanisms of authentication are: PIN and Hardware
We'll be rolling our authentication mechanisms
with the help of the Spring Framework
Is storing that information in clear text secure enough?
Probably not, we'll get back to this in a jiffy.
Right now, every single user of our application can perform any action, such as transfer funds from an account that isn't theirs
Of course, we don't want that to happen!
Spring will also help us to configure that
Who should be able to access a specific operation in our application?
To better answer this question, instead of saying user x, y and z
we can provide each one of them a role
Each user can accumulate several roles within an application
De facto security framework for JVM based languages
Reference guide
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-web</artifactId>
<version>4.2.4.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-config</artifactId>
<version>4.2.4.RELEASE</version>
<scope>runtime</scope>
</dependency>
<filter>
<filter-name>springSecurityFilterChain</filter-name>
<filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class>
</filter>
<filter-mapping>
<filter-name>springSecurityFilterChain</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:security="http://www.springframework.org/schema/security"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security.xsd">
<security:http>
<security:form-login/>
<security:intercept-url pattern="/**" access="hasAnyRole('ADMIN', 'USER')"/>
</security:http>
<security:authentication-manager>
<security:authentication-provider>
<security:user-service>
<security:user name="admin" password="password" authorities="ROLE_ADMIN"/>
<security:user name="user" password="password" authorities="ROLE_USER"/>
</security:user-service>
</security:authentication-provider>
</security:authentication-manager>
</beans>
Has a lot of built-in behaviour but this setup is rarely used
in a production ready application
To better match our needs, it is possible to override the default behaviour
Application users are declared in configuration
Good for testing purposes, bad for production
Create User and Role models and
override the default Spring Security behaviour
@Entity
@Table(name = "user")
public class User extends AbstractModel {
private String name;
private String password;
@OneToOne
private Customer customer;
@ManyToMany(fetch = FetchType.EAGER)
@JoinTable(
name = "USER_ROLE",
joinColumns = @JoinColumn(name = "USER_ID", referencedColumnName = "ID"),
inverseJoinColumns = @JoinColumn(name = "ROLE_ID", referencedColumnName = "ID")
)
private Set<Role> roles;
// getters and setters
}
@Entity
@Table(name = "role")
public class Role extends AbstractModel {
private String name;
// getters and setters
}
We need to provide Spring our own UserService implementation
<security:authentication-manager>
<security:authentication-provider user-service-ref="userServiceImpl" />
</security:authentication-manager>
The user service will be responsible for
retrieving the user data from the database
UserService has to implement the UserDetailsService interface
loadUserByUsername() locates a user by its username
@Component
public class UserServiceImpl implements UserDetailsService {
@Override
public UserDetails loadUserByUsername(String username)
throws UsernameNotFoundException {
return new User("admin", "password", Arrays.asList(
new SimpleGrantedAuthority("ROLE_ADMIN"),
new SimpleGrantedAuthority("ROLE_USER")
));
}
}
Refer to UserDetailsService and Spring Security User API docs
Right now, your user's password is stored in clear text,
which is NOT secure at all!
Sensitive information should always be hashed before stored
Let's implement our own hashing algorithm! Not really, although this would have been fun. Don't roll your own cryptography!
bcrypt is one of the most famous hashing algorithms and
a good choice for encrypting sensitive information
<security:authentication-manager>
<security:authentication-provider user-service-ref="userServiceImpl">
<security:password-encoder ref="bcryptEnconder"/>
</security:authentication-provider>
</security:authentication-manager>
<bean
name="bcryptEnconder"
class="org.springframework.security.crypto.bcrypt.BCryptPasswordEncoder"
/>
Voilà!
A common application requirement is to redirect
different types of users to different pages after login
We can achieve this by configuring Spring
to use our
Custom Authentication Success Handler
<security:http>
<security:form-login authentication-success-handler-ref="myCustomAuthenticationSuccessHandler"/>
<security:intercept-url pattern="/customer/list" access="hasRole('ADMIN')"/>
</security:http>
We can implement our custom authentication success handler by implementing SimpleUrlAuthenticationSuccessHandler
Complex rules can be transformed into a
simple boolean expression
using expression based access control
<security:http>
<security:intercept-url pattern="/customer/{customerId}/**"
access="hasRole('ADMIN') or @permissionEvaluator.hasPermission(authentication, #customerId)"/>
</security:http>
This expression prevents customers to access
other customer pages, unless they have the
ADMIN role


Use RSA-2048 whenever strong asymmetric encryption is needed.
Using strong encryption algorithms helps prevent unauthorized disclosure of sensitive information.
To use strong asymmetric encryption:
Use Java cryptographic APIs. Use the built-in Java RSA cipher with 2048 bit keys. For an example of using RSA in Java, please see the link at the bottom of the page.
Write cryptographic utility functions that use RSA-2048. Write functions that generate keys, encrypt, and decrypt data.
Use the utility functions. Use the cryptographic utility functions whenever data needs to be encrypted or decrypted using symmetric cryptography.


In some scenario could it be necessary to introduce some extern influence from the “outside" of the automaton point of view. Those kind of influences cannot be described in terms of local rules and typically require dedicated functions or procedures to be defined. An example of these kind of influences could be the lava emission from a vent of a volcano (see section [mcaFormal] for a formal definition).


|
Ocular Engine
|
Public Member Functions | |
| virtual bool | initialize () override |
| virtual void | clearBuffers (Core::Color const &clearColor=Core::Color::DefaultClearGray()) override |
| virtual void | clearDepthBuffer (float value=1.0f) override |
| virtual void | clearStencilBuffer (uint32_t value=0) override |
| virtual void | swapBuffers () override |
| virtual void | setRenderTexture (RenderTexture *texture) override |
| virtual void | setDepthTexture (DepthTexture *texture) override |
| virtual Material * | createMaterial () const override |
| virtual Viewport * | createViewport (float x, float y, float width, float height, float minDepth=0.0f, float maxDepth=1.0f) const override |
| virtual Texture * | createTexture (TextureDescriptor const &descriptor) const override |
| virtual Texture2D * | createTexture2D (TextureDescriptor const &descriptor) const override |
| virtual RenderTexture * | createRenderTexture (TextureDescriptor const &descriptor) const override |
| virtual DepthTexture * | createDepthTexture (TextureDescriptor const &descriptor) const override |
| virtual uint32_t | getMaxBoundTextures () const override |
| virtual VertexShader * | createVertexShader () const override |
| virtual GeometryShader * | createGeometryShader () const override |
| virtual FragmentShader * | createFragmentShader () const override |
| virtual PreTessellationShader * | createPreTessellationShader () const override |
| virtual PostTessellationShader * | createPostTessellationShader () const override |
| virtual UniformBuffer * | createUniformBuffer (UniformBufferType type) const override |
| virtual IndexBuffer * | createIndexBuffer () const override |
| virtual VertexBuffer * | createVertexBuffer () const override |
| virtual GPUBuffer * | createGPUBuffer (GPUBufferDescriptor const &descriptor) const override |
| virtual bool | renderMesh (Mesh *mesh, uint32_t submesh=0) override |
| virtual bool | renderBounds (Core::SceneObject *object, Math::BoundsType type) override |
| OD3D11Device * | getD3DDevice () const |
| OD3D11DeviceContext * | getD3DDeviceContext () const |
| OD3DSwapChain * | getD3DSwapChain () const |
| Public Member Functions inherited from Ocular::Graphics::GraphicsDriver | |
| virtual RenderState * | getRenderState () |
| virtual void | drawDebugLine (Math::Vector3f const &start, Math::Vector3f const &stop, Core::Color const &color, uint64_t lifetime=30000) |
| virtual void | drawDebugCircle (Math::Vector3f const ¢er, float radius, Math::Vector3f const &normal, Core::Color const &color, uint32_t segments=64, uint64_t lifetime=30000) |
| virtual void | renderDebug () |
| void | clearFrameStats () |
| FrameStats | getLastFrameStats () const |
Static Public Member Functions | |
| static bool | ConvertTextureDescriptor (TextureDescriptor const &source, D3D11_TEXTURE2D_DESC &dest) |
| static bool | ConvertTextureDescriptor (D3D11_TEXTURE2D_DESC const &source, TextureDescriptor &dest) |
Protected Member Functions | |
| virtual bool | onEvent (std::shared_ptr< Core::AEvent > event) override |
| void | resizeSwapChain (uint32_t width, uint32_t height) |
| bool | validateWindow (std::shared_ptr< Core::AWindow > window, HWND &hwnd) const |
| bool | createDeviceAndSwapChain (Core::WindowWin32 const *window, HWND const hwnd) |
| DXGI_SWAP_CHAIN_DESC | createSwapChainDescription (Core::WindowWin32 const *window) const |
| bool | fetchDeviceAndSwapChain1 (ID3D11Device *device, ID3D11DeviceContext *context, Core::WindowWin32 const *window) |
| bool | fetchDeviceAndSwapChain2 (ID3D11Device *device, ID3D11DeviceContext *context, Core::WindowWin32 const *window) |
| void | printD3DDebug () |
| Protected Member Functions inherited from Ocular::Graphics::GraphicsDriver | |
| void | addDrawCall (uint32_t numIndices) |
| Protected Member Functions inherited from Ocular::Core::AEventListener | |
| virtual bool | onEvent (std::shared_ptr< AEvent > event)=0 |
Static Protected Member Functions | |
| static bool | ValidateTextureDescriptor (TextureDescriptor const &descriptor) |
Additional Inherited Members | |
| Protected Attributes inherited from Ocular::Graphics::GraphicsDriver | |
| FrameStats | m_LastFrameStats |
| FrameStats | m_CurrFrameStats |
| Debug | m_Debug |
| RenderState * | m_RenderState |
|
overridevirtual |
Clears the backbuffer RTV, and currently bound depth/stencil buffers.
The backbuffer is cleared to the specified color value. The depth buffer is cleared to 1.0. The stencil buffer is cleared to 0.
| [in] | clearColor |
Reimplemented from Ocular::Graphics::GraphicsDriver.
|
overridevirtual |
Clears the active depth buffer to the specified value.
| [in] | value |
Reimplemented from Ocular::Graphics::GraphicsDriver.
|
overridevirtual |
Clears the active stencil buffer to the specified value.
| [in] | value |
Reimplemented from Ocular::Graphics::GraphicsDriver.
|
overridevirtual |
Creates a new API-specific implementation of the DepthTexture class.
| [in] | descriptor |
Reimplemented from Ocular::Graphics::GraphicsDriver.
|
overridevirtual |
Creates a new API-specific implementation of the FragmentShader class.
Reimplemented from Ocular::Graphics::GraphicsDriver.
|
overridevirtual |
Creates a new API-specific implementation of the GeometryShader class.
Reimplemented from Ocular::Graphics::GraphicsDriver.
|
overridevirtual |
Creates a new API-specific GPU Buffer. The type of underlying buffer depends on the parameters specified in the descriptor.
| [in] | descriptor |
Reimplemented from Ocular::Graphics::GraphicsDriver.
|
overridevirtual |
Creates a new API-specific implementation of the IndexBuffer class.
Reimplemented from Ocular::Graphics::GraphicsDriver.
|
overridevirtual |
Creates a new API-specific implementation of the Material class.
Reimplemented from Ocular::Graphics::GraphicsDriver.
|
overridevirtual |
Creates a new API-specific implementation of the PostTessellationShader class.
Reimplemented from Ocular::Graphics::GraphicsDriver.
|
overridevirtual |
Creates a new API-specific implementation of the PreTessellationShader class.
Reimplemented from Ocular::Graphics::GraphicsDriver.
|
overridevirtual |
Creates a new API-specific implementation of the RenderTexture class.
| [in] | descriptor |
Reimplemented from Ocular::Graphics::GraphicsDriver.
|
overridevirtual |
Creates a new API-specific implementation of the Texture class.
| [in] | descriptor |
Reimplemented from Ocular::Graphics::GraphicsDriver.
|
overridevirtual |
Creates a new API-specific implementation of the Texture2D class.
| [in] | descriptor |
Reimplemented from Ocular::Graphics::GraphicsDriver.
|
overridevirtual |
Creates a new API-specific implementation of the UniformBuffer class.
Reimplemented from Ocular::Graphics::GraphicsDriver.
|
overridevirtual |
Creates a new API-specific implementation of the VertexBuffer class.
Reimplemented from Ocular::Graphics::GraphicsDriver.
|
overridevirtual |
Creates a new API-specific implementation of the VertexShader class.
Reimplemented from Ocular::Graphics::GraphicsDriver.
|
overridevirtual |
Returns the maximum number of textures that can be bound to a single shader. This is primarily for use when assigning textures to materials, but can be used in other locations as well.
Reimplemented from Ocular::Graphics::GraphicsDriver.
|
overridevirtual |
Initializes the GraphicsDriver implementation.
Initialization follows the steps below:
1. Verify the window 2. Create the device, device context, and swap chain 3. Set the default render states
Reimplemented from Ocular::Graphics::GraphicsDriver.
|
overridevirtual |
Renders the bounds of the specified SceneObject
| [in] | object | |
| [in] | type |
Reimplemented from Ocular::Graphics::GraphicsDriver.
|
overridevirtual |
Renders the specified mesh and it's vertex and index buffers.
Reimplemented from Ocular::Graphics::GraphicsDriver.
|
overridevirtual |
Sets the Depth Texture to direct all depth operations towards.
| [in] | texture |
Reimplemented from Ocular::Graphics::GraphicsDriver.
|
overridevirtual |
Sets the Render Texture to direct all rendering operations towards.
| [in] | texture |
Reimplemented from Ocular::Graphics::GraphicsDriver.
|
overridevirtual |
Swaps the back and front buffers.
Reimplemented from Ocular::Graphics::GraphicsDriver.
|
staticprotected |
For the following restrictions, see the D3D11_USAGE flag values at:
https://msdn.microsoft.com/en-us/library/windows/desktop/ff476259(v=vs.85).aspx
Note: currently does not support STAGING
| Usage Flags | CPU Read | CPU Write | GPU Read | GPU Write |
|---|---|---|---|---|
| DEFAULT | Y | Y | ||
| DYNAMIC | Y | Y | ||
| IMMUTABLE | Y | |||
| STAGING | Y | Y | Y | Y |


Med Help Lines
Patient Safety
Medical Topics
Drug Finder
Eye Care
Online Textbooks
Medical Dictionary
Med Databases
Med Organizations
Med Publications
Doctor Locator
Dentist Locator
Hospital Locator
Chiropractors
Pharma Acronym & Abbreviation
Medical Libraries
Asia
Australia/New Zealand
Canada
Europe
L-G3-S
South America
United Kingdom
United States
Copyright© 1998-2003, All Health Development Corp. All rights reserved.


freeCodeCamp Survey Form
Thank you for taking the time to help us improve the platform
Name
Email
Age
Which option best describes your your current role?
Select your option
Student
Full Time Job
Full Time Learner
Prefer not to Say
Other
Would you recommend freeCodeCamp to a friend?
Definitely
Maybe
Not Sure
What is your favourite feature of freeCodeCamp?
Select your option
Challenges
Projects
Community
Open Source
What would you like to see improved? (Check all that apply)
Front End Projects
Back End Projects
Data Visualization
Challenges
Open Source Communities
Gitter Help Rooms
Videos
City Meetups
Wiki
Forum
Additional Courses
Any comments or suggestions?
Submit


complexity
Contents
1.4. complexity#
Complexity can be a useful notion for many things in statistical models. It can help answer the following questions:
can I interpret this model?
how many samples should I collect?
is my model a good fit for this problem?
model class selectiondirectionsmore stable (e.g. LOOCV, deletion, weights)interpolating estimator w/ lowest var?
set an err threshold and then look at stability
1.4.1. philosophy#
the architecture of complexity (herbert simon, 1962)
complexity takes the form of hierarchy
simon, 1981: complex systems are “made up of a large number of parts that have many interactions.”
What is complexity? (edmonds 95)
complexity is only really useful for comparisons
properties
size - makes things potentially complex
ignorance - complex things represent things we don’t understand (e.g. the brain)
minimum description size - more about information than complexity
potential problem - expressions are not much more complex than the original axioms in the system, even though they can get quite complex
potential problem - things with lots of useless info would seem more complex
some variety is necessary but not sufficient
order - we sometimes find order comes and goes (like double descent) - has a lot to do with language (audience)
defn: “that property of a language expression which makes it difficult to formulate its overall behaviour even when given almost complete information about its atomic components and their inter-relations”
language matters - what about the system are we describing?
goal matters - what outcome are interested in?
On Complexity and Emergence (standish 01)
definition close to Kolmogorov / shannon entropy
adds context dependence
kolmogorov complexity = algorithmic information complexity
problem 1: must assume a particular Universal Turing machine (which might give differing results)
problem 2: random sequences have max complexity, even though they contain no information
soln
incorporate context - what descriptions are the same?
\(C(x) = \lim _{\ell \to \infty} \log_2 N - \log_2 \omega (\ell, x)\)
where C(x) is the complexity (measured in bits), \(\ell(x)\) the length of the description, N the size of the alphabet used to encode the description and ω(ℓ,x) the size of the class of all descriptions of length less than ℓ equivalent to x.
emergence - ex. game of life
What is complexity 2 (Gell-Mann 02)
AIC - algorithmic information content - contains 2 terms
effective complexity (EC) = the length of a very concise description of an entity’s regularities
regularities are judged subjectively (e.g. birds would judge a bird song’s regularity)
2nd term relating to random features
complexity (Sporns 07)
complexity = degree to which components engage in organized structured interactions
High complexity -> mixture of order and disorder (randomness and regularity) + have a high capacity to generate emergent phenomena.
2 categories
algorithmic / mdl
natural complexity (e.g. physical complexity)
-
“the probability of producing some types of outputs is far greater when randomness operates at the level of the program describing it rather than at the level of the output itself”
“they recently reported in Royal Society Open Science that, compared to statistically random mutations, this mutational bias caused the networks to evolve toward solutions significantly faster.”
1.4.2. computational complexity#
amount of computational resource that it takes to solve a class of problem
Computational complexity (Papadimitriou)
like run times of algorithms etc. \(O(n)\)
Parameterized complexity (Downey and Fellows)
want to solve problems that are NP-hard or worse, so we isolate input into a parameter
1.4.3. bayesian model complexity#
Bayesian measures of model complexity and fit (spiegelhalter et al.)
AIC
BIC
1.4.4. statistical learning theory#
VC-dimension - measure of capacity of a function class that can be learned
cardinality of largest number of points which can be shattered
1.4.5. misc#
rashomon curves (semenova & rudin, 2019)
rashomon effect - many different explanations exist for same phenomenon
rashomon set - set of almost-equally accurate models for a given problem
rashomon ratio - ratio of volume of set of accurate models to the volume of the hypothesis space
rashomon curve - empirical risk vs rashomon ratio
rashomon elbow - maximize rashomon ratio while minimizing risk
good for model selection
bennet’s logical depth (1988) - computational resources taken to calculate the results of a minimal length problem (combines computational complexity w/ kolmogorov complexity)
Effective measure complexity (Grassberger, 1986) quantifies the complexity of a sequence by the amount of information contained in a given part of the sequence that is needed to predict the next symbol
Thermodynamic depth (Lloyd and Pagels, 1988) relates the entropy of a system to the number of possible historical paths that led to its observed state
lofgren’s interpretation and descriptive complexity
convert between system and description
kaffman’s number of conflicting constraints
Effective complexity (Gell-Mann, 1995) measures the minimal description length of a system’s regularities
Physical complexity (Adami and Cerf, 2000) is related to effective complexity and is designed to estimate the complexity of any sequence of symbols that is about a physical world or environment
Statistical complexity (Crutchfield and Young, 1989) is a component of a broader theoretic framework known as computational mechanics, and can be calculated directly from empirical data
Neural complexity (Tononi et al., 1994) - multivariate extension of mutual information that estimates the total amount of statistical structure within an arbitrarily large system.= the difference between the sum of the component’s individual entropies and the joint entropy of the system as a whole
complexity = variance of the model predictions (given that there is zero bias)
1.4.6. estimated#
optimal m estimation in high dimensions optimal loss function (optimize over different loss functions, but evaluate with L2)assumes unbiased (so variance is the mse)
1.4.7. entropy characterizations#
try to characterize functions in the prediction space
metric entropy - want functions to be close (within epsilon)
bracket entropy - function is both upper and lower bounded by bounding functions, which are within epsilon
can do this on an entire function class (e.g. all neural networks) or on a restricted subset (e.g. path during training)
Chaining Mutual Information and Tightening Generalization Bounds (asadi et al. 2018)
describing DNN paths
1.4.8. deep learning complexity#
a hessian-based complexity measure for dnnswith generalization and computation to a different form of stability
thm 3 - want function to be smooth wrt to augmented loss
complexity measure (liang et al. 2019)
1.4.9. double descent#
Reconciling modern machine learning and the bias-variance trade-off (belkin et al. 2018)
Surprises in High-Dimensional Ridgeless Least Squares Interpolation
main result of limiting risk, where \(\gamma \in (0, \infty)\):
\(R(\gamma) = \begin{cases} \sigma^2 \frac{\gamma}{1-\gamma} & \gamma < 1\\||\beta||_2^2(1 - \frac 1 \gamma) + \sigma^2 \frac{1} {\gamma - 1} & \gamma > 1\end{cases}\)
Harmless interpolation of noisy data in regression - bound on how well interpolative solns can generalize to fresh data (goes to zero with extra features)
Deep Double Descent: Where Bigger Models and More Data Hurt (nakkiran et al. 2019)
<IP_ADDRESS>. linear models#
Degrees of Freedom and Model Search (tibshirani 2014)
degrees of freedom = quantitative description of the amount of fitting performed by a given procedure
linear smoothers and additive models (buja et al. 1989) see page 469 for degrees of freedom in ridge
1.4.10. minimum description length#
mdl in linear regression: want to send y over, X is known to both sides, theta is also sent (used to pick a decoder for y)
normalized maximum likelihood (nml): use theta to make codebook, then send code
The Minimum Description Length Principle in Coding and Modeling (barron, rissanen, & yu, 98)
mdl: represent an entire class of prob. distrs. as models by a single “universal” representative model such that it would be able to imitate the behavior of any model in the class. The best model class for a set of observed data, then, is the one whose representative premits the shortest coding of the data
tradeoff: “good” prob. models for the data permit shorter code lengths
generally agrees w/ low mse
ex. encode data w/ model defined by mle estimates, quantized optimally to finite precision, then encode estimate w/ prefix code
mdl
likelihood = summarize data in accodance / model (e.g. \(P(y|x, \theta)\))
parametric complexity = summarize model params
Model Selection and the Principle of Minimum Description Length (hansen & yu 2001)
mdl: choose the model that gives the shortest description of data
description length = length of binary string used to code the data
using a prob distr. for coding/description purposes doesn’t require that it actually generate our data
basic coding
set A, code C (mapping from A to a set of codewords)
Q is a distr. on A
\(-\log_2Q\) is the code length for symbols in A
can construct such a code w/ Huffman coding
expected code length is minimized when Q = P, the true distr of our data
different forms
2-stage
mixture
predictive
normalized maximum likelihood (NML)
mdl intro (Rissanen, 2008) - scholarpedia
coding just the data would be like maximum likelihood
minimize \(\underset{\text{log-likelihood}}{-\log P(y^n|x^n;\theta)} + \underset{\text{description length}}{L(\theta)}\)
ex. OLS
if we want to send all the coefficients, assume an order and \(L(\theta) = L(p) + L(\theta_1, ... \theta_p)\)
\(L(\theta) \approx \frac p 2 \log p\)
quantization for each parameter (must quantize otherwise need to specify infinite bits of precision)
if we want only a subset of the coefficients, also need to send \(L(i_1, ..., i_k)\) for the indexes of the non-zero coefficients
minimization becomes \(\underset p \min \quad [\underset{\text{noise}}{- \log P(y^n|x^n; \hat{\theta}_{OLS})} + \underset{\text{learnable info}}{(p/2) \log n}]\)
noise - no more info can be extracted with this class of models
learnable info in the data = precisely the best model
stochastic complexity = noise + learnable info
in this case, is same as BIC but often different
modern mdl - don’t assume a model form, try to code the data as short as possible with a universal model class
often can actually construct these codes
Kolmogorov complexity \(K(x)\) = the shortest computer program (in binary) that generates x (a binary string) = the “amount of info” in x
complexity of a string x is at most its length
algorithmically random - any string whose length is close to \(|x|\)
more random = higher complexity
Minimum description length original reference \cite{rissanen1978modeling}. What is the minimum length description of the original?
MDL reviews \cite{barron1998minimum, hansen2001model}.
Book on stochastic complexity \cite{rissanen1989stochastic}
Minimum Description Length, MDL, principle for model selection, of which the original form states that the best model is the one which permits the shortest encoding of the data and the model itself
note: this type of complexity applies to the description, not the system
Look into the neurips paper on using mutual information and entropy and this paper by barron that related covering balls etc to minimax bounds
Information Theory in Probability Statistics Learning and Neural Nets (barron 97)
Information-Theoretic Asymptotics of Bayes Methods (Clarke & Barron 90)
<IP_ADDRESS>. mdl in non-linear models#
MDL-based Decision Tree Pruning (mehta et al. 95)
deep learning
high-level
most unsupervised learning can be thought of as mdl
to compress the data we must take advantage of mutual info between x and y
Learning Population Codes by Minimizing Description Length (zemel & hinton 1995)
Keeping Neural Networks Simple by Minimizing the Description Length of the Weights (hinton & van camp 1993)
The Description Length of Deep Learning Models (blier & ollivier 2018)
compress using prequential mdl
mdl for attention (lin 2019)
Lightlike Neuromanifolds, Occam’s Razor and Deep Learning (sun & nielsen 2019)
“A new MDL formulation which can explain the double descent risk curve of deep learning”
Towards Learning Convolutions from Scratch (neyshabur 2020)
uses some mdl as guiding principles
training with \(\beta\)-lasso, fc weights become very local


This small fishing port is only 10 miles south of the border from Montenegro. There are shoals off the headland to the north of the harbour. It is busy for a short time during the summer, but for the rest of the year it is very quiet.
Position 41°48'47''N, 19°35'12''E.
See Albania Formalities for detailed Clearance Information.
On approach, contact the Port Authority on VHF channel 16 and 71. They are not used to yachts and may allocate an unsuitable berth.
Visiting yachts need to use an agent for clearance here. See Albania/Clearance Agents for a list.
Last updated May 2015.
This harbour now has been dredged, so depth is not a problem. A new quay has been built, although the large ship fendering may be cause difficulties for smaller vessels. The port provides good shelter.
Security is good, but Note that the gates to the port are locked at 2200hrs and no entry or exit is permitted after that.
Water and fuel can both be delivered by tanker. Berthing fees can probably be negotiated.
Last updated June 2015.


Islamic Girls Senior High School cultivate maize farm
by Ansah Isaac Larbi, GNASuhum (E/R), Nov. 13 , GNA - Students of Islamic Girls Senior High School in Suhum in the Eastern Religion have cultivated a one acre maize farm under the Planting for Food and Jobs (PFJ) programme of the government.
This was disclosed to the Ghana News Agdncy (GNA) in an interview with Mr Emmanuel Kwadwo Gongo, a teacher of the school.
According to Mr Gongo, the Department of Agriculture of the Suhum Municipal Assembly gave the school seed maize, fertilizers and insecticides under a subsidized rates.
He said the students work on the farm daily, class by class led by the Compound Prefect after school under the supervision of the teaching staff of the school.
This is the second time the school is participating in the PFJ since it was initiated in 2017.
Mr Gongo said in the first project, the school harvested 10 bags of maize and the second one, produced six bags of maize which were used to feed the students.
He said though the school is not offering agriculture, both the students and the teaching staff are very interested in agriculture.
Mr Gongo said the school girls are much dedicated and passionately interested in farming even while in school.
He appealed to the Municipal Assembly to provide the school farming tools to support the farming activities of the school.
Mr Gongo said the school needed more land to expand the school farm and appealed to landowners to make land available to the school.
GNA