Class Cryptography
Provides methods to support file encryption and decryption functionality.
Inheritance
Namespace: DomMan
Assembly: DomMan.dll
Syntax
public static class Cryptography
Fields
cspp
Instantiate the CspParameters class.
Declaration
public static CspParameters cspp
Field Value
| Type | Description |
|---|---|
| System.Security.Cryptography.CspParameters |
rsa
RSACryptoServiceProvider variable.
Declaration
public static RSACryptoServiceProvider rsa
Field Value
| Type | Description |
|---|---|
| System.Security.Cryptography.RSACryptoServiceProvider |
Methods
AddFilePadding(String)
Writes the filePadding text to the end of an encrypted file.
Declaration
public static void AddFilePadding(string filePath)
Parameters
| Type | Name | Description |
|---|---|---|
| System.String | filePath | Path of the file to decrypt. |
DecryptFile(String)
Uses a CryptoStream object to read and decrypt the cipher text section of the FileStream encryption package, in blocks of bytes, into the FileStream object for the decrypted file.
Declaration
public static string DecryptFile(string filePath)
Parameters
| Type | Name | Description |
|---|---|---|
| System.String | filePath | Path of the file to decrypt. |
Returns
| Type | Description |
|---|---|
| System.String | The decrypted text. |
DecryptKey(String, String)
Uses the Rijndael class to decrypt the application key file.
Declaration
public static string DecryptKey(string filePath, string password)
Parameters
| Type | Name | Description |
|---|---|---|
| System.String | filePath | Application key file path. |
| System.String | password | Password to decrypt the application key file. |
Returns
| Type | Description |
|---|---|
| System.String | The decrypted string. |
EncryptFile(String, String)
Uses a CryptoStream object to read and encrypt the FileStream of the source file, in blocks of bytes, into a destination FileStream object for the encrypted file.
Declaration
public static void EncryptFile(string filePath, string plaintext)
Parameters
| Type | Name | Description |
|---|---|---|
| System.String | filePath | Path of the file to encrypt. |
| System.String | plaintext | The text to encrypt. |
EncryptKey(String, String)
Uses the Rijndael class to encrypt the application key file.
Declaration
public static void EncryptKey(string filePath, string password)
Parameters
| Type | Name | Description |
|---|---|---|
| System.String | filePath | Application key file path. |
| System.String | password | Password for encrypting the application key file. |
IsEncrypted(String)
Checks if the filePadding characters are the last characters within a file.
Declaration
public static bool IsEncrypted(string filePath)
Parameters
| Type | Name | Description |
|---|---|---|
| System.String | filePath | Path of the file to check if encrypted. |
Returns
| Type | Description |
|---|---|
| System.Boolean | true or false |
LoadKey(String, String)
Initializes an RSA object using the decrypted application key.
Declaration
public static string LoadKey(string filePath, string password)
Parameters
| Type | Name | Description |
|---|---|---|
| System.String | filePath | Application key file path. |
| System.String | password | Password to decrypt the application key file. |
Returns
| Type | Description |
|---|---|
| System.String | The exception message or key_loaded if it was successful. |
RemoveFilePadding(String)
Removes the filePadding text from the end of an encrypted file.
Declaration
public static void RemoveFilePadding(string filePath)
Parameters
| Type | Name | Description |
|---|---|---|
| System.String | filePath | Path of the file to decrypt. |


Counter Control
Counter control can be used independently or together with any filter control to print filtered items number. Any kind of filter may be used: text, path or range.
See the Pen jPList Counter Control: Dynamic by 1rosehip (@1rosehip) on CodePen.
See the Pen jPList Counter Control: Static by 1rosehip (@1rosehip) on CodePen.
HTML Structure
Counter control is defined by required data-jplist-control="counter"
data attribute. The data-group
attribute defines group of elements that should be counted.
All counters of the same group should have the same name
or data-name
attribute.
The full list of data attributes can be found here.
The counter control could be of two types: static or dynamic. The static counter is updated only once on JPlist initialization. The dynamic counter is updated on every jPList action. The dynamic is the default one.
Below is an example of dynamic counter controls together with button path controls:

<link href="jplist.styles.css" rel="stylesheet" type="text/css" />

<button
data-jplist-control="buttons-path-filter"
data-path=".title"
data-group="group1"
data-selected="true"
data-mode="radio">
Filter by title
</button>

<button
data-jplist-control="buttons-path-filter"
data-path=".desc"
data-group="group1"
data-mode="radio">
Filter by description
</button>

Title:
ddd

Description:
Lorem Ipsum has been the industry's standard dummy text ever since the 1500s.

Title:
bbb

Title:
ddd

Description:
Lorem Ipsum has been the industry's standard dummy text ever since the 1500s.

Title:
bbb

Title:
ddd

Common Data Attributes
The list of general data attributes.
| Name | Description | Values |
|---|---|---|
data-jplist-control |
Defines the counter control. | "counter" |
data-group |
Defines the group of items this control belongs to. For example, if a control has data-group="group1" then items group should have data-jplist-group="group1" data attribute. | any text value |
data-filter-type |
The type of the filter. it may be text, path or range. By default, data-name attribute has path value. | text, path or range |
data-format |
The HTML structure of the counter control with {count} placeholder that will be replaced with the actual counter value. The default value is {count}. | any text or HTML |
data-name |
The data-name attribute is used to identify the same controls in different panels. Different controls should have different data-name attributes. By default, data-name attribute has default value. | any text value |
data-mode |
static mode means that counter is updated on jPList initialization only dynamic mode means that counter is updated on every jPList action |
static or dynamic; the default is dynamic |
Path Filter Attributes
The list of data attributes for the path filter mode.
| Name | Description | Values |
|---|---|---|
data-path |
CSS selector that defines the HTML element that should be filtered | "default" keyword for the initial value or any CSS selector |
data-inverted |
If true, it's used to find all items that are not contained in the specified path. | true or false |
Text Filter Attributes
The list of data attributes for the text filter mode.
| Name | Description | Values |
|---|---|---|
data-path |
CSS selector that defines the HTML element that should be filtered | "default" keyword for the initial value or any CSS selector |
data-regex |
This optional data attribute defines characters that should be ignored during the filtering. For example, data-regex="[^a-zA-Z0-9]+" ignores all characters except English letters and numbers. | JavaScript regex |
Range Filter Attributes
The list of data attributes for the range filter mode.
| Name | Description | Values |
|---|---|---|
data-path |
CSS selector that defines the HTML element that should be filtered | "default" keyword for the initial value or any CSS selector |
data-from |
Minimal range value. | numeric value |
data-to |
Maximal range value. | numeric value |
data-min |
The minimal possible value | numeric value |
data-max |
The maximal possible value | numeric value |


Driver in court for stealing
by Caleb Kuleke, GNA
Ho, Nov. 13, GNA - Seyram Agbodemegbe, 42, a driver, has appeared before a Ho Circuit Court for allegedly misappropriating a cash sum of GH¢46,194.63 belonging to the National Investment Bank (NIB), Ho Branch.
He pleaded not guilty to the charge and was granted bail to reappear on December 3, 2019.
Prosecuting, Police Inspector Stephen Atidzon told the court presided over by Madam Priscilla Dikro-Ofori, that the complainant was Prince Kofi Adzah, Operations Manager of the Bank, whilst the accused is a driver in-charge of “On Site” delivery van of the Bank.
He said the accused also has responsibilities of conveying cashiers with monies to the Bank or collecting physical cash sales from those institutions to the Bank for deposit.
The prosecution said during the month of October, 2019, the Bank reviewed transactions of some of its top clients where it was detected that the GH¢46,194.63 collected by the accused from one of the institutions was not deposited at the Bank.
He said the accused was therefore issued with a query and he admitted dishonestly appropriating the money and a complaint was lodged with the police and he was arrested.
Inspector Atidzon said during police investigations, the accused admitted stealing the money in his cautioned statement and alleged that he used it for his father’s funeral and was therefore charged.
GNA


The relationship between mindfulness and selfawareness it that they both help us to become aware of our current state whether it be physically, mentally or emotionally.
I practised the body scan meditation. I found it surprising about which parts of my body i could bring my attention to and the sensations that I felt. Surprising in the way that these sensations were and always are there, but are 99.95% of the time they were ignored, such as becoming aware of the actue amount of effort used to keep my head up, or the minor tensions in my back muscles from my bad posture.
I really enjoyed this section. It was like reading something that i had always knew or at least felt. I like the language change from 'I am' to 'I feel' and agree that this immediately gives me the opportunity to be content my my current state of emotional or change it.


Hello and welcome to my humble corner of the internet
A Little about me:
A life long learner, constantly learning new skills in web development. While persuing my bachleors degree in Computer Science, from Queens, NYC I have also
been learning from less converional methods such as from the popular self paced website FreeCodeCamp learing more about everything from Resposive Web Design
to React.js.
Apart from just learning about web development, I have put what i have learned into practice and worked on projects to gain first hand experience with coding projects
building them from the bottom up, With most of my projects involving modern web technologies such as React.js, Node.js, etc.
My Skills and Tools
Javascript
React JS
Node JS
MongoDB
Python
HTML
CSS
Sass
Bootstrap
Projects
A React based web site that displaying random quotes from inspiring individuals from Albert Einstein to Albus Dumbledore
#React
#JS
#HTML
#CSS
#Bootstrap
A Drum sound board built using the React hooks and bootstrap, with sound files provided from freeCodeCamp
#React
#JS
#HTML
#CSS
#Bootstrap
A markdown previewer that converts github styled markdown text into html using the marked library
#React
#JS
#HTML
#CSS
#Bootstrap
A productivity based clock to focus for 25 minutes then take a 5 minute break
#React
#JS
#HTML
#CSS
#Bootstrap


Dress Diana wore when she danced with Travolta up for sale
LONDON, Nov 19 — A midnight blue velvet gown worn by Princess Diana when she danced with actor John Travolta at the White House is being put up for sale, an auction house said yesterday.
She wore the Victor Edelstein dress when she and her then husband Prince Charles attended a state dinner hosted by then president Ronald Reagan on November 9, 1985.
It was immortalised when Diana was photographed dancing with Travolta to the song You Should be Dancing from his film Saturday Night Fever.
Estimated at £250,000-£350,000 (RM1.34-1.87 million), the dress is one of three being sold by Kerry Taylor Auctions on December 9.
They also include a long-sleeved dress from 1986 by Katherine Cusack, also in midnight-blue velvet, and a Catherine Walker navy wool day dress from around 1989.
The Edelstein dress was part of a collection of outfits that Diana herself sold for charity at auction in June 1997, just weeks before she was killed in a car crash in Paris. — AFP-Relaxnews


Lori Loughlin and Mossimo Giannulli fighting new charges in college admissions scandal case
by Associated Press"Full House” actress Lori Loughlin and her fashion designer husband, Mossimo Giannulli, are contesting expanded charges against them in the college admissions bribery scandal.
Lawyers for Loughlin and Giannulli entered not guilty pleas on their behalf Tuesday to charges of conspiracy to commit federal program bribery.
LORI LOUGHLIN 'CONCERNED' BY HEFTY SENTENCE FOR PARENT WHO PLED GUILTY IN COLLEGE ADMISSIONS SCANDAL
Earlier this month, the couple waived their right to appear in federal court in Boston to answer to the latest counts.
Prosecutors added the bribery charge for 11 parents who previously pleaded not guilty in the case. Another 19 parents have pleaded guilty or agreed to do so.
Loughlin and Giannulli are accused of paying $500,000 to get their two daughters into the University of Southern California as fake athletic recruits. Their daughters no longer attend USC.
CLICK HERE TO GET THE FOX NEWS APP
The couple previously pleaded not guilty to fraud and money laundering.


{% extends 'base.html' %} {% block content %}
{{ user.email }}
You are not authenticated. Please login to access your profile.


PrInstant is a non-traditional, modular 3D printer that aims to be more time-efficient and effective than the traditional 3D printers. It uses multiple nozzles and a freely-moving printing head to simultaneously print multiple layers of an object.
The PrInstant comes in 3 parts: the printing head, the connecting tube and the material resevoir. The printing head has multiple nozzles that are activated dependent on if it is viable to use more during a print. In addition, it is made of a very flexible material able to able to simultaneously print on multiple elevations. The connecting tube is made of an elastic material allowing the printer head to move without restrictions. Finally, the material resevoir is made of a sturdy material. All 3 parts are easily connected to one another through a simple plug-and-lock system.
First, the user decides of they are going to print an online-purchased item or a model. Online retailers, such as Amazon, will notify the user if a product is "PrInstant available", which signifies if an object is compatible with the machine. If the user rather print out a pre-existing 3D model, they can use any flash device with the model's file on it and plug that into the PrInstant's USB or SD card slot. The PrInstant, rather than stacking multiple 2D layers to create a 3D object, prints multiple layers simultaneously. A PrInstant machine starts by creating a core for the object. The printing head continually keeps the core suspended in the air. It creates the item by printing around the core with multiple nozzles at different elevations. As a result, it prints multiple layers at a time. The final product is a 3D printed object potentially created 20 times quicker.
In 2014, the 3D printing market was valued at $4.1 billion and is projected to reach $44 billion by 2025. The demand for 3D printing apparatus has risen due to its potential implementations in a variety of fields. For example, 3D printed prosthetics are considered more viable as they can be fine tuned and easily modified for the user. However, the relatively slow progression in the market is due to the long waiting times to 3D print. We, at PrInstant, sternly believe that changing the traditional methods of 3D printing can enhance its viability to be used in a multitude of fields.
- 24 inches x 24 inches
- Prints all 'small' PrInstant items, as listed on amazon
- Buy and receive three boxes of printing material
- 36 inches x 36 inches
- Prints all regular sized PrInstant items and below, as listed on amazon
- Buy and receive seven boxes of printing material
- 60 inches x 60 inches
- Prints all PrInstant items, as listed on amazon
- Buy and receive twelve boxes of printing material
With their origins spanning the globe, the PrInstant team is always working as hard as they can to provide the best service they can offer you.
How does it work?
PrInstant takes materials from a reservoir and starts printing the item from the centre in a circular motion that was bought from the online e-tailor.
Where are the printers located for checkout?
As we are affliated with Amazon and other e-tailors, clicking the Add to cart will by default send the item to Amazon cart for further checkout.
How Does it Work with Amazon?
Simply go to Amazon and find an item, below should say compatible with PrInstant. To checkout with the item with PrInstant, simply click payment options and click the option with PrInstant.
Does it Work with Other E-Tailors?
Most other e-tailors have not just yet. We are working in collaboration with them to support PrInstant. The ones that do Shopify, Amazon, E-Bay, Alibaba, Walmart, and Chapters.
What Payment Types does PrInstant Support?
We support Credit Card, Debit Card, PayPal, and Bitcoin.
What if I Need to Change the Size of my Printer?
If the machine is not worn out, we will reimburse you for 80% of the value of the machine that can go towards the other printer you would like to exchange.
Where can I buy one?
You can either buy a printer from our website which will be payed on by default Amazon or buy your selected e-tailor, or you can purchase PrInstant at your local tech store.


class: center, middle # Building Microservices With Scala And Akka ## Yaroslav Tkachenko --- # About me .left-column.column-30[ ![](http://1.gravatar.com/avatar/565c77691f1676e0bab82b4881361f55?size=200) - Java, Scala - Node.js - Python - Microservices - Distributed Systems - DevOps - ... [and more](http://sap1ens.com/resume/) ] .right-column.column-70[ .text-center[ **Yaroslav (Slava) Tkachenko**, Software engineer ] .text-right[ Vancouver, BC, Canada ] **Mobify** - Senior Software Engineer, Lead, November 2016 – Present **Bench Accounting** - Director of Engineering, October 2015 – October 2016 - Engineering Lead, April 2014 – September 2015 - Software Engineer, September 2011 – March 2014 **Freelance** - Web Developer, 2007 - 2011 ] ??? Ask everyone for 10 seconds introductions Thank for participation Feel free to interrupt me anytime and ask questions --- # Agenda - Scala basics - Functional programming in Scala - Scala Futures - Actor-based programming: theory, common patterns - Actors in Akka - Supervision - Solving practical problem with Actors and Akka - Akka HTTP - Akka Clustering: sharding and singleton - Akka Persistence - Akka Streams / Alpakka - Stateless microservices with Akka - Stateful microservices with Akka ??? Mention Lightbend --- # Why Scala & Akka? .center[ ![](http://sap1ens-archive.s3-website-us-east-1.amazonaws.com/pictures/scala.jpg) ## Scala ] ??? We're not going to learn all language features. Our goal is to know enough to use Akka. --- class: center, middle # Scala basics --- # Scala basics ## Scala - General-purpose - Object-oriented - Functional - Static typing - JVM runtime ## Scala REPL \> scala --- # Scala basics - Primitive types and functions **"Primitive"** types: Byte, Char, Short, Int, Long, Float, Double, Boolean, String... Scala also supports Symbols (defined like `'name`, similar to `:symbol` in Ruby or `atom` in Erlang) Variables can be **mutable** and **immutable**: `var` and `val` Functions are defined using `def`, for example: `def length(n: String): Int` Hello World! ```scala object HelloWorld { def main(args: Array[String]): Unit = { println("Hello, world!") } } ``` ??? Symbols are interned and can be compared in constant time (O(1)) --- # Scala basics - Conditionals and loops Everything is an expression (that's why we don't need `return`)! ```scala if (x > 0) 1 else if (x == 0) 0 else -1 ``` ```scala for (i <- 1 to n) r = r * i ``` ```scala for (i <- 1 to 3; j <- 1 to 3 if i != j) print((10 * i + j) + " ") ``` ```scala for (i <- 1 to 10) yield i % 3 ``` ```scala var sum = 0 var i = 0 while (i <= 1000 && sum <= 1000) { sum += 1; i += 1 } ``` ??? - No `break` or `continue` operators. There is `import scala.util.control.Breaks._` with `breakable` and `break`, using exceptions. - For comprehensions are translated in series of flatMaps and filters --- # Scala basics - Pattern matching ```scala def matchTest(x: Any, y: Int): String = x match { case 1 => "one" case "two" => "2" case 3 | 4 | 5 => "3, 4, 5" case "ten" if y > 100 => "10" case y: Int => s"Int: $y" case _ => "idk" } ``` ```scala try { process(new URL("http://sap1ens.com")) } catch { case _: MalformedURLException => println("Bad URL: " + url) case ex: IOException => ex.printStackTrace() } ``` --- # Scala basics - Classes ```scala class Point(var x: Int, var y: Int) { def move(dx: Int, dy: Int): Unit = { x = x + dx y = y + dy } override def toString: String = "(" + x + ", " + y + ")" } ``` ```scala case class Point(x: Int, y: Int) // copy // toString // compare // ... def findX(p: Point): Int = p match { case Point(x, _) => x } ``` --- # Scala basics - Objects ```scala object Helper { def doSomething(thing: String): String = { /* ... */ } } ``` ```scala class Event(var metadata: String) object Event { def apply(): Event = new Event("") def apply(metadata: String): Event = new Event(metadata) } ``` ??? Case classes behave like they have `apply` methods defined --- # Scala basics - Traits ```scala trait One { def one(): String } trait Two { def two(): String = "two" } class Three extends One with Two { def one(): String = "one" } ``` ```scala sealed trait Account case object Chequing extends Account case object Savings extends Account case class CreditCard(user: String) extends Account ``` --- # Scala basics - Generics ```scala class Stack[T] { var elems: List[T] = Nil def push(x: T) { elems = x :: elems } def top: T = elems.head def pop() { elems = elems.tail } } ``` ```scala def count[A](l: List[A]) = l.size def count(l: List[_]) = l.size // "wildcard" ``` ```scala class Box[F <: Fruit](aFruit: F) { def fruit: F = aFruit def contains(aFruit: Fruit) = fruit.name == aFruit.name } val appleBox = new Box[Apple](new Apple) val orangeBox = new Box[Orange](new Orange) ``` --- # Scala basics - Collections .center[ ![](http://docs.scala-lang.org/resources/images/collections.png) ] --- # Scala basics - Collections ```scala List(1, 2, 3) HashMap("x" -> 24, "y" -> 25, "z" -> 26) Set(Color.red, Color.green, Color.blue) (1, "localhost") ``` - `foreach` - `map` - `filter` - `find` - `drop` / `dropWhile` - `foldRight` / `foldLeft` - `flatten` - `flatMap` - `count` - `groupBy` - ... --- # Scala basics - Collections ```scala List(1, 2, 3).foreach(element => println(element)) ``` ```scala List(1, 2, 3) foreach { element => println(element) } ``` ```scala List(1, 2, 3).foreach(println(_)) ``` ```scala List(1, 2, 3).foreach { println(_) } ``` ```scala List(1, 2, 3).foreach { println _ } ``` ```scala List(1, 2, 3).foreach(println) ``` ```scala List(1, 2, 3) foreach (println) ``` ```scala List(1, 2, 3) foreach { println } ``` ```scala List(1, 2, 3) foreach println ``` --- # Scala basics - Packages and imports ``` - com - sap1ens - data - App.scala ``` ```scala package com.sap1ens.data object Thing { def doSomething(): Unit } ``` ```scala import com.sap1ens.data.Thing Thing.doSomething() ``` --- class: center, middle # It's time to code! --- # It's time to code! Task: - Download [this file](https://raw.githubusercontent.com/jokecamp/FootballData/master/Euro%202012/Euro%202012%20stats%20TEAM.csv) - Create a case class to represent a team - Parse CSV file and keep results in memory - Find: - Team with a biggest number of goals - Teams starting with 'G' or 'P' characters and more than 5 goals - Team that completed 1200 passes. Print its name if team exists, otherwise print "Nothing found" Tips: ```scala val lines = scala.io.Source.fromFile("file.txt").getLines ``` --- # It's time to code! ```scala import scala.io.Source case class Team(name: String, goals: Int, passes: Int) val lines = Source.fromFile("/Users/sap1ens/Downloads/euro-stats.csv").getLines.toList.tail val teams: List[Team] = for(line <- lines) yield { val teamRaw = line.split(",") Team(teamRaw(0), teamRaw(1).toInt, teamRaw(12).toInt) } val teamWithBiggestGoals: Team = teams.maxBy(_.goals) val teamsStartingWithGP: List[Team] = teams .filter(team => team.name.startsWith("G") || team.name.startsWith("P")) .filter(_.goals > 5) val teamWith1200CompletedPasses: Option[Team] = teams.find(_.passes == 1200) teamWith1200CompletedPasses match { case Some(team) => println(team.name) case None => println("Nothing found") } ``` --- class: center, middle # Functional programming in Scala --- # Functional programming in Scala .center.middle[ http://www.lihaoyi.com/post/WhatsFunctionalProgrammingAllAbout.html ] --- # Functional programming in Scala Imperative recipe: ```python def make_tiramisu(eggs, sugar1, wine, cheese, cream, fingers, espresso, sugar2, cocoa): dissolve(sugar2, espresso) mixture = whisk(eggs) beat(mixture, sugar1, wine) whisk(mixture) # over steam whip(cream) beat(cheese) beat(mixture, cheese) fold(mixture, cream) assemble(mixture, fingers) sift(mixture, cocoa) refrigerate(mixture) return mixture # it's now a tiramisu ``` --- # Functional programming in Scala Functional recipe: ```python def make_tiramisu(eggs, sugar1, wine, cheese, cream, fingers, espresso, sugar2, cocoa): beat_eggs = beat(eggs) mixture = beat(beat_eggs, sugar1, wine) whisked = whisk(mixture) beat_cheese = beat(cheese) cheese_mixture = beat(whisked, beat_cheese) whipped_cream = whip(cream) folded_mixture = fold(cheese_mixture, whipped_cream) sweet_espresso = dissolve(sugar2, espresso) wet_fingers = soak2seconds(fingers, sweet_espresso) assembled = assemble(folded_mixture, wet_fingers) complete = sift(assembled, cocoa) ready_tiramisu = refrigerate(complete) return ready_tiramisu ``` ??? - No side-effects - Clear inputs and outputs - Easy to refactor and test - Easy to see dependencies (chain of calls) and missing variables --- # Functional programming in Scala .center[ ![](http://sap1ens-archive.s3-website-us-east-1.amazonaws.com/pictures/TiramisuDiagram.png) ![](http://sap1ens-archive.s3-website-us-east-1.amazonaws.com/pictures/DiagramGraph.png) ] --- # Functional programming in Scala - Functions are first class objects - Collections: ```scala def filter(p: Int => Boolean): List[Int] ``` - Immutability, laziness, tail recursion - Immutable collections, case classes, proper data types and algorithms - Lazy vals: ```scala lazy val isReady = expensiveFunc() ``` - Tail recursion ```scala @tailrec ``` - Types (*Category theory*) --- # Functional programming in Scala - Option type - Ever seen `NullPointerException`? - How about writing `foo?.bar?.baz`? - And ```javascript if(test.level1 && test.level1.level2 && test.level1.level2.level3) { // ... } ``` ? Scala's solution: ```scala sealed trait Option case object None extends Option case class Some(value: Any) extends Option ``` ```scala val absentGreeting: Option[String] = Option(null) // absentGreeting will be None val presentGreeting: Option[String] = Option("Hello!") // presentGreeting will be Some("Hello!") ``` --- # Functional programming in Scala - Option type ```scala // 1 optionVar match { case Some(i) => println(i) case None => println("That didn't work.") } // 2 if(optionVar.isDefined) { println(optionVar.get) } else { println("That didn't work.") } // 3 optionVar map { i => println(i) } getOrElse { println("That didn't work.") } ``` ??? - Missing error message / details in case of None (use Either) --- # Functional programming in Scala - Either type `Option` is great, but I also want to pass error message / details! Scala's solution: ```scala sealed trait Either case class Left(value: Any) extends Either case class Right(value: Any) extends Either ``` ```scala def getContent(url: URL): Either[String, Source] = if (url.getHost.contains("google")) Left("Requested URL is blocked for the good of the people!") else Right(Source.fromURL(url)) ``` --- # Functional programming in Scala - Either type Either is *unbiased* ```scala // 1 val content: Either[String, Iterator[String]] = getContent(new URL("http://sap1ens.com")).right.map(_.getLines()) // 2 validateName(inputName).fold { error => s"Validation failed: $error", result => s"Validation succeeded: $result" } // 3 validateName(inputName) match { case Left(error) => s"Validation failed: $error", case Right(result) => s"Validation succeeded: $result" } getContent(new URL("http://sap1ens.com")).right.toOption ``` --- class: center, middle # Scala Futures --- # Scala Futures ```scala object Future { def apply[T](body: => T)(implicit execctx: ExecutionContext): Future[T] } ``` ```scala import scala.concurrent.Future import scala.concurrent.ExecutionContext.Implicits.global import scala.util.{Success, Failure} def fetchURL(url: String): Future[String] = Future { scala.io.Source.fromURL(url).getLines.mkString } // 1 fetchURL("http://sap1ens.com") onComplete { case Success(body) => println(body) case Failure(ex) => println("Sorry, couldn't fetch that") } // 2 fetchURL("http://sap1ens.com") map { body => println(body) } recover { case ex => println("Sorry, couldn't fetch that") } ``` --- # Scala Futures ```scala Future.sequence(Seq(future1, future2, future3)) ``` ```scala val first = Future.successful("yeah") val second = Future.failed(new Exception("nope")) ``` ```scala import scala.concurrent._ import scala.concurrent.duration._ val request = fetchURL("http://sap1ens.com") Await.result(request, 5 seconds) ``` ```scala val usdQuote = Future { connection.getCurrentValue(USD) } val chfQuote = Future { connection.getCurrentValue(CHF) } val purchase = for { usd <- usdQuote chf <- chfQuote if isProfitable(usd, chf) } yield connection.buy(amount, chf) purchase onSuccess { case _ => println("Purchased " + amount + " CHF") } ``` --- # SBT - Scala Build Tool http://www.scala-sbt.org/download.html or just `brew install sbt`. Maven-like folder structure (created automatically by Intellij IDEA): ``` - project - plugins.sbt - src - main - resources - scala - com.sap1ens - Service.scala - test - resources - scala - com.sap1ens - build.sbt ``` Service.scala: ```scala object Service extends App { // put some code here... } ``` --- # SBT - Scala Build Tool build.sbt: ```scala name := """example-service""" version := "1.0" scalaVersion := "2.12.1" libraryDependencies ++= Seq( "com.typesafe" % "config" % "1.3.1" ) ``` Commands: - `sbt run` - `sbt console` - `sbt test` --- # Typesafe config https://github.com/typesafehub/config `application.conf` is a standard config file, should be located in `src/main/resources`: ``` akka { loggers = ["akka.event.slf4j.Slf4jLogger"] loglevel = "DEBUG" } db.connection_string = ${MYSQL_CONNECTION_STRING} hostname = "<IP_ADDRESS>" port = 80 ``` Use it like this: ```scala val config = ConfigFactory.load() // it's a good idea to implement it like a trait val port = conf.getInt("port") ``` --- class: center, middle # Actor systems --- # Actor systems An actor is a computational entity that, in response to a message it receives, can concurrently: - send a finite number of messages to other actors; - create a finite number of new actors; - designate the behavior to be used for the next message it receives. There is no assumed sequence to the above actions and they could be carried out in parallel. Key concepts: - Actor - Actor systems - Mailbox - Supervision strategy --- # Actor systems - Example .left-column.column-50.text-bigger[ 📬 **Rachel** ] .right-column.column-50.text-right.text-bigger[ 📬 **Alex** ] .center.text-bigger[ 📬 **Fred** ] --- # Actor systems - Supervision .center.text-bigger[ 📬 **Rachel** ] .center.text-bigger[ 💰 ⬇ ] .center.text-bigger[ 📬 **Fred** ] --- # Actor systems - Supervision .center.text-bigger[ 📬 **Rachel** ] .center.text-bigger[ 💰 ⬇ ] .center.text-bigger[ 📬 **Fred** ❌ ] --- # Actor systems - Supervision .center.text-bigger[ 📬 **Rachel** ] .center.text-bigger[ 💰 ⬇ ⬆ ] .center.text-bigger[ 📬 **Fred** ❌ ] --- # Actor systems - Supervision Strategies: - Resume the subordinate, keeping its accumulated internal state - Restart the subordinate, clearing out its accumulated internal state - Stop the subordinate permanently - Escalate the failure, thereby failing itself --- # Actor systems - Patterns Since we're dealing with messages, most of the Enterprise Integration Patterns (EIP) are applicable, for example: - Point-to-Point Channel (Actor communication) - Publish-Subscribe Channel - Scatter-Gather (Scheduler-Workers) - Pipes and Filters - ... We're going to explore more in the next section (Akka). --- # Actor systems - Scheduler-Workers .center.text-bigger[ 📬 **Scheduler** ] .center.text-bigger[ ⬇ ] .left-column.column-33.text-bigger[ 📬 **Worker1** ] .right-column.column-33.text-bigger[ 📬 **Worker2** ] .right-column.column-33.text-bigger[ 📬 **Worker3** ] --- # Actor systems - Communication styles - Fire and forget - Reply back - Pass original sender addresses - Keep addresses in the internal state --- class: center, middle # Akka --- # Akka Akka is an open-source toolkit and runtime simplifying the construction of concurrent and distributed applications on the JVM. Akka supports multiple programming models for concurrency, but it emphasizes actor-based concurrency, with inspiration drawn from Erlang. --- # Akka - Overview Why? - Concurrency and overall performance - Scalability - Clean asynchronous programming model - Resilience - Simple distributed programming --- # Akka - Reactive Manifesto .center[ ![](http://www.reactivemanifesto.org/images/reactive-traits.svg) ] --- # Akka - Concurrency Scripting languages like PHP, Ruby, Python – usually single-threaded C, C#, Java – explicit thread management Erlang, Haskell, Go – “green threads”, channels, etc. Scala: Akka – actors (“green threads”) + Java Virtual Machine --- # Akka - Actors ```scala case class Greeting(who: String) class GreetingActor extends Actor with ActorLogging { def receive = { case Greeting(who) => log.info("Hello " + who) } } val system = ActorSystem("MySystem") val greeter = system.actorOf(Props[GreetingActor], name = "greeter") greeter ! Greeting("Charlie Parker") ``` - Immutability of messages - Messages are handled one by one - Stateful actors --- class: center, middle ![](http://doc.akka.io/docs/akka/2.4/_images/actor_lifecycle1.png) --- # Akka - Actors Hooks: ```scala def preStart() def postStop() def preRestart(reason: Throwable, message: Option[Any]) def postRestart(reason: Throwable) ``` --- # Akka - Props A proper way to create Actors with parameters: ```scala object DemoActor { def props(magicNumber: Int): Props = Props(classOf[DemoActor], magicNumber) } class DemoActor(magicNumber: Int) extends Actor with ActorLogging { def receive = { case x: Int => log.info(x + magicNumber) } } val demoActor = context.actorOf(DemoActor.props(42), "demo") ``` --- # Akka - Communication ```scala class HelloActor extends Actor with ActorLogging { def receive = { case who => sender() ! "Hello, " + who } } object ConversationActor { def props(fellowActor: ActorRef): Props = Props(classOf[ConversationActor], fellowActor) } class ConversationActor(fellowActor: ActorRef) extends Actor with ActorLogging { def receive = { case "start" => fellowActor ! "it's me!" case message => log.info(message) } } val system = ActorSystem("MySystem") val helloActor = system.actorOf(Props[HelloActor]) val conversationActor = ConversationActor.props(helloActor) conversationActor ! "start" ``` --- # Akka - Communication Ask: ```scala import scala.concurrent.duration._ import akka.util.Timeout import akka.pattern.ask implicit val timeout = Timeout(5 seconds) val future = myActor ? "hello" ``` Forward: ```scala target forward message ``` Kill: ```scala import akka.actor.PoisonPill target ! PoisonPill ``` --- # Akka - Behavior ```scala class HotSwapActor extends Actor { import context._ def angry: Receive = { case "foo" => sender() ! "I am already angry?" case "bar" => become(happy) } def happy: Receive = { case "bar" => sender() ! "I am already happy :-)" case "foo" => become(angry) } def receive = { case "foo" => become(angry) case "bar" => become(happy) } } ``` --- # Akka - Behavior ```scala import akka.actor.Stash class ActorWithProtocol extends Actor with Stash { def receive = { case "open" => unstashAll() context.become({ case "write" => // do writing... case "close" => unstashAll() context.unbecome() case msg => stash() }, discardOld = false) // stack on top instead of replacing case msg => stash() } } ``` --- # Akka - Routing ```scala val router: ActorRef = context.actorOf(RoundRobinPool(5).props(Props[Worker]), "router") ``` - RoundRobinRoutingLogic - RandomRoutingLogic - SmallestMailboxRoutingLogic - BroadcastRoutingLogic - ScatterGatherFirstCompletedRoutingLogic - TailChoppingRoutingLogic - ConsistentHashingRoutingLogic --- # Akka - Mailboxes ``` control-aware-dispatcher { mailbox-type = "akka.dispatch.UnboundedControlAwareMailbox" } ``` ```scala val a = system.actorOf(Props(classOf[Logger], this).withDispatcher("control-aware-dispatcher")) ``` - **UnboundedMailbox** - SingleConsumerOnlyUnboundedMailbox - NonBlockingBoundedMailbox - UnboundedControlAwareMailbox - UnboundedPriorityMailbox - UnboundedStablePriorityMailbox - BoundedMailbox - BoundedPriorityMailbox - BoundedStablePriorityMailbox - BoundedControlAwareMailbox --- # Akka - Event Stream ```scala system.eventStream.subscribe(listenerActor, classOf[Message]) ``` ```scala system.eventStream.publish(Message("Hi!")) ``` For example: ```scala import akka.actor.AllDeadLetters system.eventStream.subscribe(listener, classOf[AllDeadLetters]) ``` or local Pub/Sub. --- # Akka - Supervision ```scala class Supervisor extends Actor { override val supervisorStrategy = OneForOneStrategy(maxNrOfRetries = 10, withinTimeRange = 1 minute) { case _: ArithmeticException ⇒ Resume case _: NullPointerException ⇒ Restart case _: Exception ⇒ Escalate } val worker = context.actorOf(Props[Worker]) def receive = { case n: Int => worker forward n } } ``` --- # Akka - Actors & Futures ```scala implicit val timeout = Timeout(5 seconds) val future = actor ? msg // 1 future onComplete { case Success(result) => doSomethingOnSuccess(result) case Failure(failure) => doSomethingOnFailure(failure) } // 2 future.map(result => doSomething(result)) // 3 for { r <- future } yield doSomething(r) ``` --- class: center, middle # Blocking Is Bad --- # Akka and Application Architecture *We have actors. What are supposed to do with them? MVC? SOA? Something else? How about DI? How do we structure actor-based apps?* Let's talk about: - **DDD** (Domain-Driven Design) - **EIP** (Enterprise Integration Patterns) --- # Akka and Application Architecture .left-column.column-45.text-right[ ![](https://images-na.ssl-images-amazon.com/images/I/5146azDZjmL._SX358_BO1,204,203,200_.jpg) ] .right-column.column-45[ ![](https://images-na.ssl-images-amazon.com/images/I/51dGyfBYvuL._SX382_BO1,204,203,200_.jpg) ] --- # Akka and Application Architecture .left-column.column-50[ Domain Driven Design: - Value Object - Entity - Service - Repository - Domain Event - Event vs Command - ... ] .right-column.column-50[ Enterprise Integration Patterns: - Message - Point-to-Point - Publish-Subscribe - Pipes and Filters - Content-Based Router - Message Filter - Scatter-Gather - ... ] --- # Akka and Application Architecture .center[ ![](http://sap1ens-archive.s3-website-us-east-1.amazonaws.com/pictures/workshop_akka_design_example.png) ] --- # Akka and Application Architecture ```scala object SynchronizationService { // data model sealed trait DocumentMetadata { def version: String } case class DocumentMetadataV2(...) extends DocumentMetadata case class DocumentMetadataV3(...) extends DocumentMetadata // actions case class SaveDocument(...) case class StartSync(connection: Connection) private case class CreateStatement(...) private case class FindOrCreateStatementAccount(...) private case class CreateStatementAccount(...) private case class SaveAndUploadStatement(...) // ... ``` --- # Akka and Application Architecture ```scala // ... // results sealed trait DeliveryResult case object Delivered extends DeliveryResult case object Failed extends DeliveryResult case object NotFound extends DeliveryResult private case class SyncFailure(...) def props(s3Service: ActorRef, exportService: ActorRef, connectionsResource: ConnectionsResourceLike, filestorePath: String): Props = Props(classOf[SynchronizationService], s3Service, exportService, connectionsResource, filestorePath) } ``` --- # Akka and Application Architecture ```scala class SynchronizationService(s3Service: ActorRef, exportService: ActorRef, connectionsResource: ConnectionsResourceLike, filestorePath: String) extends Actor with ActorLogging { def receive: Receive = { case StartSync(connection) => // ... case SaveDocument(metadata, file) => // ... case action @ CreateStatement(...) => // ... case FindOrCreateStatementAccount(action, connection, receiver) => // ... case CreateStatementAccount(action, connection, receiver) => // ... case SaveAndUploadStatement(action, connection, statementAccount, receiver) => // ... case SyncFailure(error, action, receiver, deliveryResult) => // ... case UploadResultOk(statement, receiver) => // ... case UploadResultFailure(error, receiver) => // ... } } ``` --- # Akka and Application Architecture ```scala trait CoreActors extends ConfigHolder { // helpers val integrationApiClient = new APIClient() val s3Service = system.actorOf(S3Service.props(config.getString("s3.bucket"), config.getString("s3.accessKey"), config.getString("s3.secretKey"), config.getInt("s3.maxRetries") ), "S3Service") val fileManager = system.actorOf(FileManager.props(config.getString("filestore"), s3Service), "FileManager") // services val exportService = system.actorOf(ExportService.props( integrationApiClient, fileManager, Duration(config.getInt("export.interval"), MINUTES), config.getBoolean("export.enabled"), config.getInt("export.maxRetries") ), "StatementsExportService") val synchronizationService = system.actorOf(SynchronizationService.props( s3Service, exportService, ConnectionsResource, config.getString("filestore") ), "SynchronizationService") } ``` --- # Akka and Application Architecture We've just created Application **Core**. We can add more interfaces like **HTTP** and **messaging queues**. - HTTP, we'll look into Akka HTTP soon - Messaging queues, let's use Akka Camel --- # ~~Akka~~ Apache Camel http://camel.apache.org "The most unknown coolest library out there" JM (c) *Apache Camel offers you the interfaces for the EIPs, the base objects, commonly needed implementations, debugging tools, a configuration system, and many other helpers which will save you a ton of time when you want to implement your solution to follow the EIPs.* ```java from("direct:report") .to("file:target/reports/?fileName=report.txt") from("twitter://search?...") .to("websocket:camel-tweet?sendToAll=true") from("netty-http:http://<IP_ADDRESS>:8080") .to("direct:name") from("jms:invoices") .setBody() .groovy("new Invoice(request.body,currentTimeMillis())") .to("mongodb:mongo?...operation=insert") ``` Hundreds of components for everything! https://camel.apache.org/components.html --- # Akka Camel Akka Camel Consumer: ```scala class OrdersConsumer extends Consumer { def endpointUri = "activemq:queue:orders" def receive = { case msg: CamelMessage => { /* ... */ } case _ => { /* ... */ } } } ``` Akka Camel Producer: ```scala class OrdersProducer extends Actor with Producer { def endpointUri = "activemq:queue:orders" } val orders = system.actorOf(Props[Orders]) orders ! Order(...) ``` --- # Akka and Application Architecture General guidelines: - Set of Actors as a core layer - Additional layers for integration, persistence, etc. - Akka HTTP for REST API - Akka Camel for messaging queues --- # Akka and Application Architecture Hexagonal architecture: .center[ ![](http://alistair.cockburn.us/get/2301) ] --- # Akka and Application Architecture Hexagonal architecture: .center[ ![](http://alistair.cockburn.us/get/2304) ] --- class: center, middle # It's time to code, again! --- # It's time to code, again! Task: https://github.com/sap1ens/scala-akka-workshop (use this to start). Let's build a [job queue](https://en.wikipedia.org/wiki/Job_queue)! In our case we're going to have `JobExecutor` actor that accepts incoming job requests and executes them using pool of `JobWorker` actors. If one of the `JobWorker` actors fails `JobExecutor` should restart failed job. Also `JobWorker`s should communicate results to `JobExecutor`. A job itself can be anything (read or write a file, calculate something, etc.), but I suggest to download a file from specified URL and save it to local disk with provided path. Ideally we should introduce a percent of failures, so we can see restarts as well. A few important things: - All Akka dependencies are already included in build.sbt - You can run the application with `sbt run` - Use case classes to represent jobs - If you add `ActorLogging` trait to your actor you'll be able to print logging messages with `log.info(...)`, `log.error(...)`, etc. --- # It's time to code, again! Solution: https://github.com/sap1ens/scala-akka-workshop/tree/solution/src/main/scala/com/sap1ens/workshop Solution with Futures: https://github.com/sap1ens/scala-akka-workshop/tree/solution-with-futures/src/main/scala/com/sap1ens/workshop --- class: center, middle # Akka Streams --- # Akka Streams http://www.reactive-streams.org Reactive Streams is an initiative to provide a standard for asynchronous stream processing with non-blocking back pressure. This encompasses efforts aimed at runtime environments (JVM and JavaScript) as well as network protocols. In summary, Reactive Streams is a standard and specification for Stream-oriented libraries for the JVM that - process a potentially unbounded number of elements - in sequence, - asynchronously passing elements between components, - with mandatory non-blocking backpressure. --- # Akka Streams - Source - Flow - Sink and also Graphs! ```scala implicit val actorSystem = ActorSystem("akka-streams-example") implicit val materializer = ActorMaterializer() val helloWorldStream: RunnableGraph[NotUsed] = Source.single("Hello world") .via(Flow[String].map(s => s.toUpperCase())) .to(Sink.foreach(println)) helloWorldStream.run() ``` --- # Akka Streams .left-column.column-33[ Source stages: - fromIterator - apply - single - repeat - cycle - tick - fromFuture - fromCompletionStage - unfold - unfoldAsync - empty - maybe - ... ] .left-column.column-33[ Simple Flow processing stages: - map - mapConcat - statefulMapConcat - filter - filterNot - collect - grouped - sliding - scan - scanAsync - fold - foldAsync - ... ] .left-column.column-33[ Sink stages: - head - headOption - last - lastOption - ignore - cancelled - seq - foreach - foreachParallel - onComplete - lazyInit - queue - ... ] --- # Akka Streams ```scala implicit val system = ActorSystem("Sys") implicit val materializer = ActorMaterializer() val LoglevelPattern = """.*\[(DEBUG|INFO|WARN|ERROR)\].*""".r FileIO.fromPath(Paths.get("src/main/resources/logfile.txt")) // parse chunks of bytes into lines .via(Framing.delimiter(ByteString(System.lineSeparator), maximumFrameLength = 512, allowTruncation = true)) .map(_.utf8String) .map { case line @ LoglevelPattern(level) => (level, line) case line @ other => ("OTHER", line) } .groupBy(5, _._1) // group them by log level .fold(("", List.empty[String])) { case ((_, list), (level, line)) => (level, line :: list) } .mapAsync(parallelism = 5) { // write lines of each group to a separate file case (level, groupList) => Source(groupList.reverse) .map(line => ByteString(line + "\n")).runWith(FileIO.toPath(Paths.get(s"target/log-$level.txt"))) } .mergeSubstreams .runWith(Sink.onComplete { case Success(_) => system.terminate() case Failure(e) => println(s"Failure: ${e.getMessage}") system.terminate() }) ``` --- # Akka Streams ```scala val g = RunnableGraph.fromGraph(GraphDSL.create() { implicit builder: GraphDSL.Builder[NotUsed] => import GraphDSL.Implicits._ val in = Source(1 to 10) val out = Sink.ignore val bcast = builder.add(Broadcast[Int](2)) val merge = builder.add(Merge[Int](2)) val f1, f2, f3, f4 = Flow[Int].map(_ + 10) in ~> f1 ~> bcast ~> f2 ~> merge ~> f3 ~> out bcast ~> f4 ~> merge ClosedShape }) ``` --- class: center, middle # Alpakka --- # Alpakka https://github.com/akka/alpakka http://developer.lightbend.com/docs/alpakka/current/ *Welcome to the home of the Alpakka initiative, which harbours various Akka Streams connectors, integration patterns, and data transformations for integration use cases.* Integrations so far: - amqp - cassandra - dynamodb - file - ftp - hbase - jms - mqtt - s3 --- # Alpakka File: ```scala val fs = FileSystems.getDefault val lines: Source[String, NotUsed] = scaladsl.FileTailSource.lines( path = fs.getPath(path), maxLineSize = 8192, pollingInterval = 250.millis ) lines.runForeach(line => println(line)) ``` ```scala val fs = FileSystems.getDefault val changes = DirectoryChangesSource(fs.getPath(path), pollInterval = 1.second, maxBufferSize = 1000) changes.runForeach { case (path, change) => println("Path: " + path + ", Change: " + change) } ``` --- # Alpakka AMQP: ```scala val queueName = "amqp-conn-it-spec-simple-queue" val queueDeclaration = QueueDeclaration(queueName) val amqpSink = AmqpSink.simple( AmqpSinkSettings(DefaultAmqpConnection).withRoutingKey(queueName).withDeclarations(queueDeclaration) ) val input = Vector("one", "two", "three", "four", "five") Source(input).map(s => ByteString(s)).runWith(amqpSink) ``` ```scala val amqpSource = AmqpSource( NamedQueueSourceSettings(DefaultAmqpConnection, queueName).withDeclarations(queueDeclaration), bufferSize = 10 ) val result = amqpSource.map(_.bytes.utf8String).take(input.size).runWith(Sink.seq) ``` --- # Alpakka S3 source code fragment: ```scala // ... SplitAfterSize(chunkSize)(Flow.apply[ByteString]) .via(getChunkBuffer(chunkSize)) //creates the chunks .concatSubstreams .zipWith(requestInfo) { case (chunkedPayload, (uploadInfo, chunkIndex)) => //each of the payload requests are created val partRequest = HttpRequests.uploadPartRequest(uploadInfo, chunkIndex, chunkedPayload.data, chunkedPayload.size, region) (partRequest, (uploadInfo, chunkIndex)) } .mapAsync(parallelism) { case (req, info) => Signer.signedRequest(req, signingKey).zip(Future.successful(info)) } // ... ``` --- class: center, middle # Akka HTTP --- # Akka HTTP - Reincarnation of the famous [Spray](http://spray.io) library - Akka-native (based on Akka Streams) - Very fast and performant - Very strict implementation - Has Server and Client APIs - Server APIs: - Low-level (dealing with connections and streams) - High-level (using routing and directives) - Client APIs: - Low-level Connection API - Low-level Host API - High-level Request API --- # Akka HTTP - High-level Server Example ```scala import akka.actor.ActorSystem import akka.http.scaladsl.Http import akka.http.scaladsl.model._ import akka.http.scaladsl.server.Directives._ import akka.stream.ActorMaterializer object WebServer { def main(args: Array[String]) { implicit val system = ActorSystem("my-system") implicit val materializer = ActorMaterializer() val route = path("hello") { get { complete(HttpEntity(ContentTypes.`text/html(UTF-8)`, "
Say hello to akka-http
")) } } Http().bindAndHandle(route, "localhost", 8080) } } ``` --- # Akka HTTP - More Routes ```scala val route = { path("orders") { authenticateBasic(realm = "admin area", myAuthenticator) { user => get { encodeResponseWith(Deflate) { complete { // marshal custom object with in-scope marshaller retrieveOrdersFromDB } } } ~ post { // decompress gzipped or deflated requests if required decodeRequest { // unmarshal with in-scope unmarshaller entity(as[Order]) { order => complete { // ... write order to DB "Order received" } } } } } } ``` --- # Akka HTTP - More Routes ```scala val route = { // extract URI path element as Int pathPrefix("order" / IntNumber) { orderId => pathEnd { put { // form extraction from multipart or www-url-encoded forms formFields(('email, 'total.as[Money])).as(Order) { order => complete { // complete with serialized Future result (myDbActor ? Update(order)).mapTo[TransactionResult] } } } } ~ path("items") { get { // parameters to case class extraction parameters(('size.as[Int], 'color ?, 'dangerous ? "no")) .as(OrderItem) { orderItem => // ... route using case class instance created from // required and optional query parameters complete("") // hide } } } } ``` --- # Akka HTTP - More Routes ```scala val route = { pathPrefix("documentation") { // optionally compresses the response with Gzip or Deflate // if the client accepts compressed responses encodeResponse { // serve up static content from a JAR resource getFromResourceDirectory("docs") } } ~ path("oldApi" / Remaining) { pathRest => redirect("http://oldapi.example.com/" + pathRest, MovedPermanently) } } ``` --- # Akka HTTP - The Routing Tree ```scala val route = a { b { c { ... // route 1 } ~ d { ... // route 2 } ~ ... // route 3 } ~ e { ... // route 4 } } ``` - Route 1 will only be reached if directives `a`, `b` and `c` all let the request pass through. - Route 2 will run if `a` and `b` pass, `c` rejects and `d` passes. - Route 3 will run if `a` and `b` pass, but `c` and `d` reject. --- # Akka HTTP - Serialization ```scala import akka.http.scaladsl.server.Directives import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport import spray.json._ final case class Item(name: String, id: xrLong) final case class Order(items: List[Item]) trait JsonSupport extends SprayJsonSupport with DefaultJsonProtocol { implicit val itemFormat = jsonFormat2(Item) implicit val orderFormat = jsonFormat1(Order) // contains List[Item] } class MyJsonService extends Directives with JsonSupport { val route = get { pathSingleSlash { complete(Item("thing", 42)) // will render as JSON } } ~ post { entity(as[Order]) { order => // will unmarshal JSON to Order val itemsCount = order.items.size val itemNames = order.items.map(_.name).mkString(", ") complete(s"Ordered $itemsCount items: $itemNames") } } } ``` --- # Akka HTTP - Low-level Server Example ```scala implicit val system = ActorSystem() implicit val materializer = ActorMaterializer() implicit val executionContext = system.dispatcher val serverSource = Http().bind(interface = "localhost", port = 8080) val requestHandler: HttpRequest => HttpResponse = { case HttpRequest(GET, Uri.Path("/"), _, _, _) => HttpResponse(entity = HttpEntity(ContentTypes.`text/html(UTF-8)`, "Hello world!")) case HttpRequest(GET, Uri.Path("/ping"), _, _, _) => HttpResponse(entity = "PONG!") case HttpRequest(GET, Uri.Path("/crash"), _, _, _) => sys.error("BOOM!") case r: HttpRequest => r.discardEntityBytes() // important to drain incoming HTTP Entity stream HttpResponse(404, entity = "Unknown resource!") } val bindingFuture: Future[Http.ServerBinding] = serverSource.to(Sink.foreach { connection => println("Accepted new connection from " + connection.remoteAddress) connection handleWithSyncHandler requestHandler }).run() ``` --- # Akka HTTP / Streams Akka HTTP is streaming *all the way through*, which means that the back-pressure mechanisms enabled by Akka Streams are exposed through all layers–from the TCP layer, through the HTTP server, all the way up to the user-facing `HttpRequest` and `HttpResponse` and their `HttpEntity` APIs. This has surprising implications if you are used to non-streaming / not-reactive HTTP clients. Specifically it means that: “*lack of consumption of the HTTP Entity, is signaled as back-pressure to the other side of the connection*”. This is a feature, as it allows one only to consume the entity, and back-pressure servers/clients from overwhelming our application, possibly causing un-necessary buffering of the entity in memory. --- class: center, middle # Akka Persistence --- # Akka Persistence - Overview .center[ ![](http://sap1ens-archive.s3-website-us-east-1.amazonaws.com/pictures/akka-persistence.jpg) ] --- # Akka Persistence - Overview - Event Sourcing - Journal - Persistent Actor - Persistent View Has plugins for JDBC (MySQL, Postgres, ...), MongoDB, Cassandra, Kafka, Redis and more. --- # Akka Persistence - Example ```scala case class Cmd(data: String) case class Evt(data: String) case class ExampleState(events: List[String] = Nil) { def updated(evt: Evt): ExampleState = copy(evt.data :: events) override def toString: String = events.reverse.toString } class ExamplePersistentActor extends PersistentActor { override def persistenceId = "sample-id-1" var state = ExampleState() def updateState(event: Evt): Unit = state = state.updated(event) val receiveRecover: Receive = { case evt: Evt => updateState(evt) case SnapshotOffer(_, snapshot: ExampleState) => state = snapshot } val receiveCommand: Receive = { case Cmd(data) => persist(Evt(data))(updateState) case "snap" => saveSnapshot(state) case "print" => println(state) } } ``` --- # Akka Persistence - Example ```scala class MyView extends PersistentView { override def persistenceId: String = "some-persistence-id" override def viewId: String = "some-persistence-id-view" def receive: Receive = { case payload if isPersistent => // handle message from journal... case payload => // handle message from user-land... } } ``` --- # Akka Persistence - Configuration ```scala libraryDependencies += "com.github.dnvriend" %% "akka-persistence-jdbc" % "2.6.12" ``` ``` akka { persistence { journal.plugin = "jdbc-journal" snapshot-store.plugin = "jdbc-snapshot-store" } } akka-persistence-jdbc { slick { driver = "slick.driver.PostgresDriver" db { driver = "org.postgresql.Driver" url = "jdbc:postgresql://localhost/db" user = "postgres" password = "" keepAliveConnection = on numThreads = 20 queueSize = 100 } } tables { // ... } // ... } ``` --- class: center, middle # Akka Clustering --- # Akka Clustering - Overview .center[ ![](http://sap1ens-archive.s3-website-us-east-1.amazonaws.com/pictures/node-ring.png) ] .left-column.column-50[ - Cluster - Node - Leader ] .right-column.column-50[ - Gossip protocol - Seed nodes - Failure Detector ] --- # Akka Clustering - Overview .center[ ![](http://doc.akka.io/docs/akka/2.4/_images/member-states.png) ] --- # Akka Clustering - Setup ``` akka { actor { provider = cluster } remote { netty.tcp { hostname = "<IP_ADDRESS>" port = 0 } } cluster { seed-nodes = [ "akka.tcp://ClusterSystem@<IP_ADDRESS>:2551", "akka.tcp://ClusterSystem@<IP_ADDRESS>:2552" ] } } ``` --- # Akka Clustering - Setup ```scala package sample.cluster.simple import akka.cluster.Cluster import akka.cluster.ClusterEvent._ import akka.actor.ActorLogging import akka.actor.Actor class SimpleClusterListener extends Actor with ActorLogging { val cluster = Cluster(context.system) override def preStart(): Unit = cluster.subscribe(self, InitialStateAsEvents, classOf[MemberEvent]) override def postStop(): Unit = cluster.unsubscribe(self) def receive = { case MemberUp(member) => log.info("Member is Up: {}", member.address) case MemberRemoved(member, previousStatus) => log.info("Member is Removed: {} after {}", member.address, previousStatus) case _: MemberEvent => } } ``` --- # Akka Clustering - Setup Real world example: https://github.com/sap1ens/akka-cluster-consul ```scala val cluster = Cluster(system) val selfAddress = cluster.selfAddress val serviceAddresses = ConsulAPI.getServiceAddresses // http://doc.akka.io/docs/akka/2.4/scala/cluster-usage.html // // When using joinSeedNodes you should not include the node itself except for the node // that is supposed to be the first seed node, and that should be placed first // in parameter to joinSeedNodes. val serviceSeeds = serviceAddresses filter { address => address != selfAddress || address == serviceAddresses.head } cluster.joinSeedNodes(serviceSeeds) cluster registerOnMemberUp { // init system } ``` --- # Akka Clustering - Actors 3 nodes, 3 actors, but 9 messages: ``` INFO - Cluster Node [akka.tcp://demo-system@<IP_ADDRESS>:9102] - Member is Up [akka.tcp://demo-system@<IP_ADDRESS>:9101] INFO - Cluster Node [akka.tcp://demo-system@<IP_ADDRESS>:9101] - Member is Up [akka.tcp://demo-system@<IP_ADDRESS>:9102] INFO - Cluster Node [akka.tcp://demo-system@<IP_ADDRESS>:9103] - Member is Up [akka.tcp://demo-system@<IP_ADDRESS>:9103] INFO - Cluster Node [akka.tcp://demo-system@<IP_ADDRESS>:9101] - Member is Up [akka.tcp://demo-system@<IP_ADDRESS>:9101] INFO - Cluster Node [akka.tcp://demo-system@<IP_ADDRESS>:9103] - Member is Up [akka.tcp://demo-system@<IP_ADDRESS>:9102] INFO - Cluster Node [akka.tcp://demo-system@<IP_ADDRESS>:9102] - Member is Up [akka.tcp://demo-system@<IP_ADDRESS>:9103] INFO - Cluster Node [akka.tcp://demo-system@<IP_ADDRESS>:9101] - Member is Up [akka.tcp://demo-system@<IP_ADDRESS>:9103] INFO - Cluster Node [akka.tcp://demo-system@<IP_ADDRESS>:9102] - Member is Up [akka.tcp://demo-system@<IP_ADDRESS>:9102] INFO - Cluster Node [akka.tcp://demo-system@<IP_ADDRESS>:9103] - Member is Up [akka.tcp://demo-system@<IP_ADDRESS>:9101] ``` --- # Akka Clustering - Singleton Why? - Single point of responsibility for certain cluster-wide consistent decisions, or coordination of actions across the cluster system - Single entry point to an external system - Single master, many workers - Centralized naming service, or routing logic But: - Cluster singleton may quickly become a performance bottleneck - You can not rely on the cluster singleton to be non-stop available — e.g. when the node on which the singleton has been running dies, it will take a few seconds for this to be noticed and the singleton be migrated to another node - In the case of a network partition appearing in a Cluster that is using Automatic Downing (see Auto Downing docs for Downing), it may happen that the isolated clusters each decide to spin up their own singleton, meaning that there might be multiple singletons running in the system, yet the Clusters have no way of finding out about them (because of the partition) --- # Akka Clustering - Singleton Create Singleton: ```scala system.actorOf( ClusterSingletonManager.props( singletonProps = Props(classOf[Consumer], queue, testActor), terminationMessage = End, settings = ClusterSingletonManagerSettings(system)), name = "consumer") ``` Call Singleton: ```scala val consumer = system.actorOf( ClusterSingletonProxy.props( singletonManagerPath = "/user/consumer", settings = ClusterSingletonProxySettings(system)), name = "consumerProxy") consumer ! SomeMessage ``` --- # Akka Clustering - Sharding Features: - One of the most powerful Akka features! - Allows to route messages across nodes in a cluster using a sharding function (actually two) - You don't need to know the physical location of an actor - cluster will forward message to a remote node if needed - Uses Akka Persistence internally (or brand-new Distributed Data) Concepts: - Coordinator - Shard Region - Entity Entities (actors) are "activated" by receiving a first message and can be "passivated" using `context.setReceiveTimeout`. ??? - Cluster has only one coordinator as a singleton - Ideal number of shards = max(nodes) * 10 - Every shard contains set of entities --- # Akka Clustering - Sharding Counter interface: ```scala case object Increment case object Decrement final case class Get(counterId: Long) final case class EntityEnvelope(id: Long, payload: Any) case object Stop final case class CounterChanged(delta: Int) ``` --- # Akka Clustering - Sharding Counter implementation: ```scala class Counter extends PersistentActor { context.setReceiveTimeout(120.seconds) override def persistenceId: String = "Counter-" + self.path.name var count = 0 def updateState(event: CounterChanged): Unit = count += event.delta override def receiveRecover: Receive = { case evt: CounterChanged ⇒ updateState(evt) } override def receiveCommand: Receive = { case Increment ⇒ persist(CounterChanged(+1))(updateState) case Decrement ⇒ persist(CounterChanged(-1))(updateState) case Get(_) ⇒ sender() ! count case ReceiveTimeout ⇒ context.parent ! Passivate(stopMessage = Stop) case Stop ⇒ context.stop(self) } } ``` --- # Akka Clustering - Sharding Create a region on every node: ```scala val counterRegion: ActorRef = ClusterSharding(system).start( typeName = "Counter", entityProps = Props[Counter], settings = ClusterShardingSettings(system), extractEntityId = extractEntityId, extractShardId = extractShardId) ``` Sharding functions: ```scala val extractEntityId: ShardRegion.ExtractEntityId = { case EntityEnvelope(id, payload) ⇒ (id.toString, payload) case msg @ Get(id) ⇒ (id.toString, msg) } val numberOfShards = 100 val extractShardId: ShardRegion.ExtractShardId = { case EntityEnvelope(id, _) ⇒ (id % numberOfShards).toString case Get(id) ⇒ (id % numberOfShards).toString } ``` --- class: center, middle # Akka Cluster Sharding + Persistence = ❤ --- class: center, middle # Stateless microservices with Akka --- # Stateless microservices with Akka **Stateless** application: application that doesn't keep any state in *runtime*, but uses external services instead. - External service: database, cache, API, etc. - Examples: most of the web apps are stateless or build to be stateless (Django, Rails, Express, etc.). Even enterprise frameworks like Spring (Java). --- # Stateless microservices with Akka Stateless microservice can be represented as a black-box with a set of inputs and outputs. Inputs: - HTTP API (Akka HTTP) - Messaging queue (Akka Camel / Akka Streams) - RPC Outputs: - Database (Slick) - Cache - Other API - Messaging queue - ... --- # Stateless microservices with Akka .center[ ![](http://sap1ens-archive.s3-website-us-east-1.amazonaws.com/pictures/stateless-microservices.png) ] --- # Stateless microservices with Akka http://www.jamesward.com/2014/12/03/java-doesnt-suck-youre-just-using-it-wrong > Sticky sessions and server state are usually one of the best ways to kill your performance and resilience. Session state (in the traditional Servlet sense) makes it really hard to do Continuous Delivery and scale horizontally. If you want a session cache use a real cache system – something that was designed to deal with multi-node use and failure. e.g. Memcache, ehcache, etc. In-memory caches are fast but hard to invalidate in multi-node environments and are not durable across restarts – they have their place, like calculated / derived properties where invalidation and recalculation are easy. > > Web apps should move state to the edges. UI-related state should live on the client (e.g. cookies, local storage, and in-memory) and in external data stores (e.g. SQL/NoSQL databases, Memcache stores, and distributed cache clusters). Keep those REST services 100% stateless or else the state monster will literally eat you in your sleep. --- # Stateless microservices with Akka .center[ ![](http://sap1ens-archive.s3-website-us-east-1.amazonaws.com/pictures/nodes-with-lb.png) ] --- class: center, middle # Stateful microservices with Akka --- # Stateful microservices with Akka **Stateful** application: application that keeps internal state in *runtime*, instead of relying on external services. - Examples: actors *can* be stateful, so Akka and other actor-based systems (Erlang/OTP, Orleans) can be stateful. But it's also possible to create stateful applications in Node.js or Python, for example. "Building Scalable Stateful Services" by Caitie McCaffrey https://www.youtube.com/watch?v=H0i_bXKwujQ --- # Stateful microservices with Akka Benefits: - Data locality => low latency, fast processing - Sticky consistency => "simple" and "cheap" consistency without using consensus protocols Biggest challenges: - High availability - Scaling out --- # Stateful microservices with Akka .center[ ![](http://sap1ens-archive.s3-website-us-east-1.amazonaws.com/pictures/nodes-with-lb.png) Remember stateless: every request is routed randomly (by LB). Chaotic DB reads. ] --- # Stateful microservices with Akka .center[ ![](http://sap1ens-archive.s3-website-us-east-1.amazonaws.com/pictures/consistent-hashing.png) Now stateful: consistent selection of reads. ] --- # Stateful microservices with Akka .center[ ![](http://sap1ens-archive.s3-website-us-east-1.amazonaws.com/pictures/node-ring.png) **Akka Cluster Sharding + Akka Persistence / Distributed Data** ] --- # Stateful microservices with Akka Akka Cluster Sharding: - Consistent hashing for all requests based on user-defined function - Automatic forwarding (from local to remote and vice versa) Akka Persistence: - Keeping internal state - Easy and fast recovery (journal + snapshots) - Event-sourcing built-in Akka Distributed Data [experimental]: - One of the options for Akka Persistence backend - Supports CRDTs like counters, sets and maps --- # Stateful microservices with Akka Real-world example: "tracker" service. Complex state machine for multiple entities. Inputs: - Domain Events via messaging queue (Akka Camel) Outputs: - HTTP API (Akka HTTP) - Websockets (Akka HTTP) Every entity has a **clientId** and they never intersect - it's a perfect use-case for sharding (clientId as a sharding key). --- # Stateful microservices with Akka ```scala // model trait Message { def clientId: Long } trait EventMessage extends Message trait StateMessage extends Message case class BankAccountMessage( clientId: Long, eventType: EventName, ledgerId: String, ledgerName: String, months: Seq[Month] ) extends EventMessage // sharding configuration // ideal number of shards is 10x of number of nodes in future val numberOfShards = config.getInt("tracker.sharding.numShards") val extractEntityId: ShardRegion.ExtractEntityId = { case msg: Message => (msg.clientId.toString, msg) } val extractShardId: ShardRegion.ExtractShardId = { case msg: Message => (msg.clientId % numberOfShards).toString } ``` --- # Stateful microservices with Akka ```scala object TrackerService { case class TrackerData( accounts: Map[String, BankAccount] = Map[String, BankAccount](), states: Map[Month, ClientState] = Map[Month, ClientState]() ) { def isEmpty = accounts.isEmpty && states.isEmpty } } class TrackerService extends PersistentActor { private var state = TrackerData() private def handleMessage(message: EventMessage, notifySubscribers: Boolean) { val maybeUpdatedState = message match { case b: BankAccountMessage => handleBankMessage(b) case c: ClientMessage => handleClientMessage(c) case _ => None } maybeUpdatedState.foreach { updatedState => updateState(message.clientId, updatedState, notifySubscribers) } } private def updateState(clientId: Long, updatedState: TrackerData) = { state = state.copy( accounts = (state.accounts ++ updatedState.accounts).filterNot(_._2.deleted), states = state.states ++ updatedState.states ) } } ``` --- # Stateful microservices with Akka Passivation is a big deal: ```scala class TrackerService extends PersistentActor { context.setReceiveTimeout(1 hour) override def receiveCommand: Receive = { case Stop => saveSnapshot(state) context.stop(self) } } ``` Custom logic for snapshots: ```scala override def persist[A](event: A)(handler: (A) => Unit) = { super.persist(event)(handler) if (lastSequenceNr > 0 && lastSequenceNr % snapshotThreshold == 0) { self ! CreateSnapshot } } ``` --- # Summary - Scala is great! And I hope you think the same 😉. It can be very concise, but flexible and efficient. Especially if you follow FP principles - Actors and Futures are better concepts to use for concurrency - Akka is a great toolkit for building distributed systems - Try various things! Akka can look very complicated, but it's not that scary when you try it - Read papers ;-) --- class: center, middle # We did it! Thank you! Please answer quick servey: https://goo.gl/forms/Lh9MmiicFxTTdAv43


Start new round
Open in ID editor
Accept solution
This is not a bug
Skip this bug
project on github
Press
Start new round
to load a new set of bugs to fix!


---
layout: page
title: SOLID principles for data science
subtitle: Part 1 - Single-Responsibility Principle
use-site-title: true
---
SOLID Principles
SOLID are a set of 5 principles intended to help the developers produce maintenable, readable and flexible
code:
single-responsibility principle
open–closed principle
Liskov substitution principle
interface segregation principle:
dependency inversion principle
Single-Responsibility Principle
The main description of the SRP is often the following: "There should never be more than one reason for a class to
change."
Depending on the person describing the principle the interpretation can be different.
It is often interpreted as "every class should have only one responsibility." which is a really broad definition
that can be difficult to apply in a real environment.
I personally prefer the definition given by "Clean Architecture" of Robert Martin. He say that a module should only
depend on a given group of person (in a typical company, a group of person may be given by occupations, like
marketing, accounting, finance, ...).
For instance, a module should have to change only if accounting change the way they are calculating some value of
interest, but should never change if marketing change any of their process.
This principle is about people.
Code example
In the previous implementation, the module mix the way the values of the report are computed, which is a
responsability that can be attributed for example to finance or accounting and the way the report is presented, which
can be for example the responsability of marketing to show the good results of their marketing campaing to the CEO.
A good implementation would be to separate the two "responsability" in different module.


I recently graduated with a masters in Business Analytics and Information Management from Purdue University. I have 3 years of experience as a Business Analyst and an MBA from BITS Pilani, India.
Interested in advertising, marketing analytics and data science. Open to Business analyst, Data scientist and Data analyst roles.
GPA: 3.7 / 4
GPA: 7.4 / 10
GPA: 6.9 / 10
Apart from being an analyst, I enjoy camping and hiking. Weekends are best spent exploring the countryside and stargazing. When forced indoors, I follow a number of comedy and sci-fi genre movies and television shows.

Sessions Lead for BITS Pilani Consulting club.
Organized guest lectures and seminars, facilitated case solving sessions for club members on cases involving market entry, profitability, market expansion scenarios
Coordinator for the Branding team at BITS Pilani
Led a team of 8 to design and implement digital marketing strategies for master’s degree admissions. Achieved a 35% YOY increase in number of applicants


Best practices for working with AWS Lambda functions
The following are recommended best practices for using AWS Lambda:
Function code
-
Separate the Lambda handler from your core logic. This allows you to make a more unit-testable function. In Node.js this may look like:
exports.myHandler = function(event, context, callback) { var foo = event.foo; var bar = event.bar; var result = MyLambdaFunction (foo, bar); callback(null, result); } function MyLambdaFunction (foo, bar) { // MyLambdaFunction logic here }
-
Take advantage of execution environment reuse to improve the performance of your function. Initialize SDK clients and database connections outside of the function handler, and cache static assets locally in the
/tmp
directory. Subsequent invocations processed by the same instance of your function can reuse these resources. This saves cost by reducing function run time.To avoid potential data leaks across invocations, don’t use the execution environment to store user data, events, or other information with security implications. If your function relies on a mutable state that can’t be stored in memory within the handler, consider creating a separate function or separate versions of a function for each user.
-
Use a keep-alive directive to maintain persistent connections. Lambda purges idle connections over time. Attempting to reuse an idle connection when invoking a function will result in a connection error. To maintain your persistent connection, use the keep-alive directive associated with your runtime. For an example, see Reusing Connections with Keep-Alive in Node.js.
-
Use environment variables to pass operational parameters to your function. For example, if you are writing to an Amazon S3 bucket, instead of hard-coding the bucket name you are writing to, configure the bucket name as an environment variable.
-
Control the dependencies in your function's deployment package. The AWS Lambda execution environment contains a number of libraries such as the AWS SDK for the Node.js and Python runtimes (a full list can be found here: Lambda runtimes). To enable the latest set of features and security updates, Lambda will periodically update these libraries. These updates may introduce subtle changes to the behavior of your Lambda function. To have full control of the dependencies your function uses, package all of your dependencies with your deployment package.
-
Minimize your deployment package size to its runtime necessities. This will reduce the amount of time that it takes for your deployment package to be downloaded and unpacked ahead of invocation. For functions authored in Java or .NET Core, avoid uploading the entire AWS SDK library as part of your deployment package. Instead, selectively depend on the modules which pick up components of the SDK you need (e.g. DynamoDB, Amazon S3 SDK modules and Lambda core libraries
). -
Reduce the time it takes Lambda to unpack deployment packages authored in Java by putting your dependency
.jar
files in a separate /lib directory. This is faster than putting all your function’s code in a single jar with a large number of.class
files. See Deploy Java Lambda functions with .zip or JAR file archives for instructions. -
Minimize the complexity of your dependencies. Prefer simpler frameworks that load quickly on execution environment startup. For example, prefer simpler Java dependency injection (IoC) frameworks like Dagger
or Guice , over more complex ones like Spring Framework . -
Avoid using recursive code in your Lambda function, wherein the function automatically calls itself until some arbitrary criteria is met. This could lead to unintended volume of function invocations and escalated costs. If you do accidentally do so, set the function reserved concurrency to
0
immediately to throttle all invocations to the function, while you update the code.
Function configuration
-
Performance testing your Lambda function is a crucial part in ensuring you pick the optimum memory size configuration. Any increase in memory size triggers an equivalent increase in CPU available to your function. The memory usage for your function is determined per-invoke and can be viewed in AWS CloudWatch Logs. On each invoke a
REPORT:
entry will be made, as shown below:REPORT RequestId: 3604209a-e9a3-11e6-939a-754dd98c7be3 Duration: 12.34 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 18 MB
By analyzing the
Max Memory Used:
field, you can determine if your function needs more memory or if you over-provisioned your function's memory size.To find the right memory configuration for your functions, we recommend using the open source AWS Lambda Power Tuning project. For more information, see AWS Lambda Power Tuning
on GitHub. To optimize function performance, we also recommend deploying libraries that can leverage Advanced Vector Extensions 2 (AVX2). This allows you to process demanding workloads, including machine learning inferencing, media processing, high performance computing (HPC), scientific simulations, and financial modeling. For more information, see Creating faster AWS Lambda functions with AVX2
. -
Load test your Lambda function to determine an optimum timeout value. It is important to analyze how long your function runs so that you can better determine any problems with a dependency service that may increase the concurrency of the function beyond what you expect. This is especially important when your Lambda function makes network calls to resources that may not handle Lambda's scaling.
-
Use most-restrictive permissions when setting IAM policies. Understand the resources and operations your Lambda function needs, and limit the execution role to these permissions. For more information, see AWS Lambda permissions.
-
Be familiar with Lambda quotas. Payload size, file descriptors and /tmp space are often overlooked when determining runtime resource limits.
-
Delete Lambda functions that you are no longer using. By doing so, the unused functions won't needlessly count against your deployment package size limit.
-
If you are using Amazon Simple Queue Service as an event source, make sure the value of the function's expected invocation time does not exceed the Visibility Timeout value on the queue. This applies both to CreateFunction and UpdateFunctionConfiguration.
-
In the case of CreateFunction, AWS Lambda will fail the function creation process.
-
In the case of UpdateFunctionConfiguration, it could result in duplicate invocations of the function.
-
Metrics and alarms
-
Use Working with AWS Lambda function metrics and CloudWatch Alarms instead of creating or updating a metric from within your Lambda function code. It's a much more efficient way to track the health of your Lambda functions, allowing you to catch issues early in the development process. For instance, you can configure an alarm based on the expected duration of your Lambda function invocation in order to address any bottlenecks or latencies attributable to your function code.
-
Leverage your logging library and AWS Lambda Metrics and Dimensions to catch app errors (e.g. ERR, ERROR, WARNING, etc.)
Working with streams
-
Test with different batch and record sizes so that the polling frequency of each event source is tuned to how quickly your function is able to complete its task. BatchSize controls the maximum number of records that can be sent to your function with each invoke. A larger batch size can often more efficiently absorb the invoke overhead across a larger set of records, increasing your throughput.
By default, Lambda invokes your function as soon as records are available in the stream. If the batch that Lambda reads from the stream only has one record in it, Lambda sends only one record to the function. To avoid invoking the function with a small number of records, you can tell the event source to buffer records for up to five minutes by configuring a batch window. Before invoking the function, Lambda continues to read records from the stream until it has gathered a full batch, or until the batch window expires.
-
Increase Kinesis stream processing throughput by adding shards. A Kinesis stream is composed of one or more shards. Lambda will poll each shard with at most one concurrent invocation. For example, if your stream has 100 active shards, there will be at most 100 Lambda function invocations running concurrently. Increasing the number of shards will directly increase the number of maximum concurrent Lambda function invocations and can increase your Kinesis stream processing throughput. If you are increasing the number of shards in a Kinesis stream, make sure you have picked a good partition key (see Partition Keys) for your data, so that related records end up on the same shards and your data is well distributed.
-
Use Amazon CloudWatch on IteratorAge to determine if your Kinesis stream is being processed. For example, configure a CloudWatch alarm with a maximum setting to 30000 (30 seconds).


Try out our Premium Member services: Virtual Legal Assistant, Query Alert Service and an ad-free experience. Free for one month and pay only if you like it.
IN THE HIGH COURT OF GUJARAT AT AHMEDABAD CIVIL APPLICATION No 5132 of 1996 in SPECIAL CIVIL APPLICATIONNo 2922 of 1995 -------------------------------------------------------------- KAVERI HOLDINGS PVT LTD Versus AJIT PADIWAL -------------------------------------------------------------- Appearance: MR BH CHHATRAPATI for Petitioner MR AD PADIVAL for Respondent No. 1 -------------------------------------------------------------- CORAM : MR.JUSTICE B.C.PATEL and MR.JUSTICE M.S.SHAH Date of Order: 11/02/97 ORAL ORDER
The Unit is closed to-day and by this application has requested the Court to permit the unit to go for trial production.
We have heard the learned Counsel for the applicant as well as the GPCB. We have perused the report submitted by the Modi Committee and the affidavit alongwith other record in this behalf. Reading the affidavit it appears that the unit is keen to carry out the recommendations suggested by the Modi Committee. Mr.Nanavati requested the Court that in view of this the Unit may be given permission for a trial production. He fairly submitted that if the Unit is inspected by the NEERI, the Unitwill have the benefit of the said Report. He submitted that the Unit will deposit Rs,1,00,000/-for the said purpose.
Mr.Nanavati has submitted that the Unit had agreed to pay Rs.ten lacs for the purpose of utilization towards general welfare, improvement of environment, monitoring charges, etc. by 23.10.1996. However, on account of the difficulties the Unit could not make the payment. Mr.Nanavati however, requested that the Unit will make payment of Rs.9 lacs within one week from to-day and for payment of Rs.one lac he requested for time and stated that within three weeks that amount of Rs.one lac will be paid. The amount of Rs.one lac which is required to be paid for NEERI inspection, will be deposited within one week. Accordingly, time of one week is granted as aforesaid.
Mr.Baxi learned Advocate pointed out that fresh consent application is not given by the Unit. Mr.Nanavati states that the application is already forwarded but in case it is not forwarded, the Unit will forward the same immediately.
Learned Counsel submitted that the petitioner shall file undertaking within the time stipulated to the effect that the amount of Rs.9 lacs and the amount of Rs.1 lac towards the inspection of NEERI shall be paid as aforesaid and the unit shall make consent application, if not made so far. Undertaking to be filed on or before 14-2-1997. After filing the Undertaking the Office will send intimation to the GEB to reconnect the electric supply at the cost of the Unit and subject to clearance of all past dues.
In the facts and circumstances of the case, we permit the Unit to go for trial production on general conditions and on condition that the amount as aforesaid is deposited and the application for consent is made to the GPCB immediately. The GPCB will process the consent application in accordance with law.
S.O. to 19-2-1997.
(B.C.Patel J. ) (M.S.Shah J. ) IN THE HIGH COURT OF GUJARAT AT AHMEDABAD SPECIAL CIVIL APPLN NO. 5132 of 1996 ANNEXURE TO ORDER DATED: 11.2.1997.
GENERAL TERMS & CONDITIONS FOR CHEMICAL INDUSTRIES & OTHERS PRODUCTS:
1. The manufacturing/processing activities of the Unit shall be confined to the product/s mentioned in its application to the GPCB for consent under the Water (Prevfention and Control of Pollution) Act,1974; Air(Prevention and Control of Polution ) Act, 1981 and in its application for authorisation under the Hazardous Wastes (Management and Handling ) Rules, 1989. The Unit shall not manufacture or process any other product. If the product/process falls in Schedule I or II to the Scheme of Environmental Audit the Unit shall also abide by the said Scheme. The application for consent/authorization shall mean the last application before this order.
PROHIBITION AGAINST REMOVAL/SALE OF GOODS:
2. The unit shall undertake production/ processing activities only for the limited purpose of satisfying the GPCB and other monitoring agencies that the unit meets with all the norms prescribed by the GPCB for the treatment of the effluent.Hence the Unit shall not remove the goods from the factory premises nor shall the unit issue any Excise Gate-pass without prior permission of the Court.
EFFLUENT TREATMENT:
3(a). The unit shall operate fully and efficiently all its effluent treatment plant/s and shall close down all its manufacturing/processing activities whenever the effluent treatment plant/s or any part thereof are fully or partly non-operational for any reason whatsoever (whether for maintenance/repairs/electricity failure or otherwise) and shall not restart such activities unless and until all the effluent treatment plants of the unit are fully operational.
(b) The Unit shall have and operate all the requisite equipments/facilities for prevention and control of air pollution and shall operate the same. The unit shall also have stackmonitoring facilities.Whenever the equipments/facilities for prevention and control of air pollution are fully or partly nonfunctional, the unit shall close down all its manufacturing /processing activities and shall not restart its manufacturing/processing activities unless and until all its air pollution protection and control equipments and facilities including stock monitoring facilities are fully operational.
(c). The Unit shall make all the requisite arrangements for the safe storage and handling of solid waste including impervious flooring and leachate collection and the unit shall store and handle solid waste in accordance with the provisions of the relevant rules in this behalf.
EFFLUENT DISPOSAL:
4. The unit shall have and use only one outlet for the discharge of its effluent into ECP and no effluent shall be discharged without requisite treatment and without meeting with the GPCB norms. Such outlet shall be near the front gate /entrance of the unit. The unit shall not keep any bypass line or system, or loose or flexible pipe for discharging effluent outside or even for transporting treated or untreated effluent within the factory premises, within effluent treatment plants or in the compound of the unit.
DISPLAY BOARDS AND LIGHTING:
(5). The Unit shall, within one week from the date of the order:
(a). put up at the entrance and prominent places boards prominently displaying the name of the unit, particulars of the products/process and the names of the Proprietor/Partners/Directors of the Unit, the electricity consumer number and the name of the electricity consumer as on the record of the AEC/GEB.
(b). put up a board displaying the details mentioned in para 6(a) hereinafter.
(c)make adequate lighting arrangements all around the Effluent Treatment Plants and also above the boards mentioned in both the above clauses;
INFORMATION TO BE SUPPLIED TO THE GPCB & OTHERS:
6(a). The Unit shall submit, within one week from the date of the order, to the GPCB and the Chairman of the Coordinating Committee appointed by the Court, the site plan of the unit indicating the location of manufacturing/processing plants as also the effluent treatment plants, and also a separate plan indicating the channel through which water/ effluent passes from different stages of manufacturing/processing and the effluent treatment process right upto the stage of its final outlet. Such plans shall also be displayed by the Unit on a Board of adequate size within its compound and near its effluent treatment plant/s.
(b). The Unit shall, within one week from the date of the order, furnish to the GPCB and the Chairman of the Coordinating Committee details about the raw materials and the nature of manufacturing process/ reactions.
(c). The unit shall supply to the GPCB the figures of production and consumption of electricity and water for each day during the period of trial production, though such figures shall be supplied on weekly basis. The unit shall supply separate figures for consumption of electricity for running the effluent treatment plant/s by having a separate meter/sub-meter for such effluent treatment plant/s. The number of Units consumed by operating the diesel generating sets,if any, shall also be supplied to the GPCB on weekly basis.
(d) The Unit shall also supply to the GPCB, within one week from the date of the order, the documents regarding monthly production and consumption of electricity in the six months prior to the date on which the unit was closed.
ENTRY & INSPECTION :
7(a). The unit shall permit the officers/employees of the GPCB/AMC/Govt., Member/s of the Committee appointed by this Court, Member/s of the monitoring Committee of the Association/s of the Industries to enter the factory premises and to inspect and take samples from the unit at any time without any prior intimation. Any delay in giving any of the above persons entry into the factory premises or any part thereof or effluent treatment plants shall entail closure of the unit. All the watchmen /security personnel of the unit shall be immediately appraised of the above.
(b). It shall be open to the GPCB through general instructions or circulars and to the GPCB Officer/s inspecting the Unit to give on the spot instructions regarding location of the outlet and/or any other appropriate directions regarding effluent treatment plants, their operations and processes and disposal channel and disposal system. The Unit shall comply with all such instructions, whether general or special.
FUTURE PAYMENTS :
8(a). The unit shall deposit an amount of Rs.4000/on each occasion when sampleS are taken by the GPCB. The unit shall initially deposit with the GPCB Rs.4000/- for the first sample, within one week from the date of the order.
(b). The unit shall pay all the instalments of the contribution/s for laying,maintaining and repairing effluent drainage pipeline/s as and when demanded by a public authority or by the Association of Industries in the area or as and when ordered by this Court.
(c). The Unit shall pay the Association of Industries in the area and the body managing the Solid Waste Dumping site for the industries in the area all the charges and contributions as and when demanded by them for preparing the dumping site, removing solid wazste to such site and for connected activities.
(d). The unit shall pay such amount as may be determined by the Court/Govt./GPCB/Association for compensation and remedial measures for the affected areas and people and for monitoring, for the period after commencement of trial production.
IN THE EVENT OF CLOSURE:
9. When electricity supply or water supply is disconnected in future on account of noncompliance with the GPCB norms or on account of the closure order which may be passed by this Court or by the Government/GPCB under any statutory provisions relating to environmental protection and prevention and control of pollution;
(a) the unit shall not use any diesel generating set or any other alternative source of energy;
(b) the Unit shall pay wages to its workers regularly every month or at such shorter intervals as per the central/practice followed so far.
...
sharma


Get current time:
$ curl "/current"
2018-12-15T02:06:59.439+01:00
This service is protected by a
rate
limiter.
If too many requests are received, HTTP error 429 Too Many Requests
is sent.

This microservice is using the circuit breaker pattern to handle service outage. Find out what happens if you try to reach this API endpoint while the backend service is not running.