Rapid prototypers, manufacturers, local supply chains, companies, service providers based in and around South Wales can help the NHS fill short term gaps in the supply chain and design and manufacture critical items and crucial parts.
To see what projects are running, get involved in one of the projects or just support a project, please contact the helpdesk or email<EMAIL_ADDRESS>SWARM is The South Wales Additive and Rapid Manufacturing Consortia. It is here to help the NHS source and supply items it critically needs to address Covid-19. SWARM is an initiative to help industry meet the demands of the primary supply chain and to pass intelligence to the NHS about supplies, materials and ideas that industry, manufacturing and design might have, that could help their efforts.
No. We only educate the supply chain as to the need and match that need with an offer.
We match needs to offers and then provide business intelligence to industry, healthcare workers and procurement teams.
If you want to approach NHS Wales nationally you should visit the Life Science Hub and register your details there. https://lshubwales.com/call-industry-collaboration-fight-coronavirus. If you are not already an NHS Wales or other healthcare organisation approved supplier, you may find it difficult to offer help directly to the hospitals or other organisations. You may have problems providing help, because of established rules. There are well established crisis and safety protocols to protect patients and staff. Making direct offers to help, however well intentioned, can overwhelm staff and procurement teams at a time when they are managing a crisis. We are here to provide a summary of offers to the teams when they are best placed to use them and share information about specific needs with the rest of the industry community too.
SWARM aims to:
• respond to health board’s Covid-19 strategies by rapidly gathering information on the availability of needed equipment to feed into the regional supply chain.
• organise the local manufacturing and design expertise to respond to requests for novel design solutions needed ahead of the supply chain.
• organise a volunteer effort to support both the health board’s needs and frontline services that lie outside the health board’s remit (care homes etc).
• convey information gained to the health boards in an efficient and timely way that supports and does not disrupt existing supply chains.
SWARM gathers information about available capacity, materials, products and knowledge from the industry, manufacturing and design communities in response to specific needs identified by the frontline health service community.
This resource is made possible, sponsored and supported by the following organisations (in alphabetical order).
Assessmint, Datalytix, DVLA, Industry Wales, Swansea University, TechHub Swansea, University of Wales Trinity St. David, Vortex IOT, Welsh Automotive Forum, Welsh Government
Working with Welsh Government and NHS Wales, SWARM is The South Wales Additive and Rapid Manufacturing Consortia. It is here to help the NHS source and supply items it critically needs to address Covid-19.
SWARM is an initiative to help industry meet the demands of the primary supply chain and to pass intelligence to the NHS about materials and ideas industry, manufacturing and design might have, that could help their efforts.


We have now covered all the important technical parts of the 10k steps challenge application: how to build web APIs, web applications, edge services, use databases and perform event stream processing. By using Vert.x, asynchronous and reactive programming we can expect the set of services that form the application to be reactive: scalable as workloads grow, and resilient as failures happen.
Are the services that we just built actually reactive? Let us discover now through testing and experimentation, and see where we can make improvements towards being reactive.
To do that, we will use load testing tools to stress services and measure latencies. We will then add failures using chaos testing tools to see how this impacts the service behaviors, and we will discuss several options to fix the problems that we will identify. You will be able to apply this methodology in your own projects, too.
This chapter is extensively based on experiments, hence we need to generate some workloads to assess how the application copes with demanding workloads and failures. There are many load testing tools around and it is not always easy to pick one. Some tools are very good at stressing a service with a specific request (e.g., "what is the latency when issuing 500 requests per second to /api/hello"). Some tools provide more flexibility by offering scripting capabilities (e.g., "simulate a user that logs in, then adds items to a cart, then perform a purchase"). And finally, some tools do all of that, but the reported metrics may be inaccurate due to how such tools are implemented.
I have chosen 2 popular and easy-to-use tools that we will use in this chapter:
The 2 tools can be used together, or not: Locust allows us to simulate a representative workload of users interacting with the application, while Hey give us precise metrics of how specific HTTP endpoints behave under stress.
Both Locust and Hey work on Linux, macOS and Windows. As usual if you are a Windows user then I recommend that you use the Windows Subsystem for Linux.
Before we run any load testing tool I would like to discuss a few points that have to be considered to get representative results, and most importantly, interpret them with care.
First of all when you run 10k steps application as outlined in chapter 7 all services are running locally, while the third party middleware and services are running in Docker containers. This means that everything is actually running on the same machine, avoiding real network communications. For instance when the user profile service talks to MongoDB then it goes through virtual network interfaces, but it never reaches an actual network interface so there is no fluctuating latency or data loss. We will use other tools later in this chapter to simulate network problems, and get a more precise understanding of how our services behave.
There is a good chance that you will perform these experiments on your laptop or desktop. Keep in mind that a real server is different from your workstation on both hardware and software configurations, so you will likely perform tests with lower workloads than what the services could actually cope with in a production setting. For instance when we use PostgreSQL directly from a container, we haven’t done any tuning like we would do in a production setting. More generally running the middleware services from containers is convenient for development purposes, but we would run them differently in production, with or without containers. Also note that we will be running the Vert.x-based services without any JVM tuning. In a production setting you’d need to at least adjust memory settings and tune the garbage collector.
Each service will run as a single instance, where verticles will also be single instances. They have all been designed to work with multiple instances, but deploying, say, 2 instances of the ingestion service would also require deploying a HTTP reverse proxy to distribute traffic between the 2 instances.
 You can perform the tests on a single machine if that is more convenient to you, but keep in mind that:
The results that I present in this chapter are based on experiments conducted with 2 Apple MacBook laptops which hardly qualify as production-grade servers. I am also using a domestic WiFi network, which is not as good as an Ethernet wired connection, especially when it comes to having a stable latency. Finally macOS has very low limits on the number of file descriptors that a process can open (256), so I have to raise them with the ulimit command to run the services and the load testing tools, otherwise errors unrelated to the services code can happen just because too many connections have been opened. I will show you how to do that, and you will likely have to use it as well for running the experiments.
Locust is a tool for generating workloads by simulating users interacting with a service. You can use it to make demonstrations, tests and measuring performance.
You will need a recent version of Python on your machine. If you are new to Python then you can read [PythonBasics] or one of the many tutorials online. At the time of this writing I am using Python 3.8.2. You can install Locust by running pip install locust on the command line, where pip is the standard Python package manager.
The Locust file that we are using is called locustfile.py and it can be found in the part2-steps-challenge/load-testing folder of the Git repository. Here we will be simulating the following user behaviors, as illustrated in figure 12.1.
a user fetches a JWT token on the first request after having been registered, then periodically makes requests where either:
This activity covers most of the services, as ingesting triggers event exchanges between most services, while API queries triggers calls to the activity and user profile services.
The locustfile.py file defines 2 classes: UserBehavior defines the tasks performed by a user, while UserWithDevice runs these tasks with a random delay between 0.5 and 2 seconds. This is a relatively short delay between requests to increase the overall number of requests per second. There are 2 parameters for running a test with Locust:
As described in chapter 7, you need to run the container services with Docker Compose from the part2-steps-challenge folder using docker-compose up in a terminal, then run all Vert.x-based services in another terminal. You can use foreman start if you have foreman installed, or else you can run all services using the commands in Procfile.
Listing 12.1 provides the command to perform an initial "warm-up" run.
$ cd part2-steps-challenge/load-testing $ ulimit -n 10000 #1 $ locust --no-web \ #2 --host http://<IP_ADDRESS> \ #3 --clients 50 --hatch-rate 1 --run-time 3m #4
It is important to do such a warm-up run because the JVM running the various services need to have some workload before they can start to run code efficiently. After that we can run a bigger workload to get a first estimation of how our services go. Listing 12.2 gives the command to make a test for 5 minutes with 150 clients and a hatch rate of 2 new users per second.
$ locust --no-web \ --host http://<IP_ADDRESS> \ --clients 150 --hatch-rate 2 --run-time 5m \ --csv data/locust-run #1
Let us run the experiment, and collect results. We get various metrics on each type of request, such as the average response time, the minimum / maximum times, the median time, etc. What is actually an interesting metric is the latency given a percentile.
Let’s take the example of the latency at the 80th percentile: this is the maximum latency observed for 80% of the requests. If that latency is of 100ms, then it means that 80% of the requests took less than 100ms. Similarly, if the 95th percentile latency is of 150ms, then it means that 95% of the requests took at most 150ms. The 100th percentile reveals the worst case observed.
When measuring performance we are often interested in the latencies between the 95th and 100th percentiles. Suppose that latency at the 90th percentile is 50ms, but 3s at the 95th percentile and 20s at the 99th percentile. In such a case we clearly have a performance problem because we observe a large share of bad latencies. By contrast observing a latency of 50ms at the 90th percentile and 70ms at the 99th percentile shows a service with a very consistent behavior.
Looking at latency distributions is more telling of a service behavior under load than looking at the average latency. What we are actually interested is not the best cases but those were we observed the worst results. Figure 12.2 shows us latency report over a run that I did with 150 users over 5 minutes.
The plot contains values at the 95th, 98th, 99th and 100th percentiles. The reported latencies are under 200ms at the 99th percentile for all requests, which sound reasonable for a run with imperfect conditions and no tuning. The 100th percentile values show us the worst response times that have been observed, and they are all under 500ms.
We could increase the number of users to stress the application even more, but we are not going to do the precise load testing with Locust. If you raise the number of users you will quickly start seeing increasing latencies and errors being raised. This is not due to the application under test but a limitation of Locust at the time of the writing:
For accurate load testing we thus have to use another tool, and Hey is a good one.
Locust is still a great tool to produce a small workload and even automate a demo of the project. Once it has started and simulates users, you can connect to the dashboard web application and see it be updated live.
Hey is a much simpler tool than Locust as it cannot be scripted, and it focuses on stressing a HTTP endpoint. It is however an excellent tool for getting accurate measures on an endpoint under stress.
We are still going to use Locust on the side to simulate a small number of users. This will generate some activity in the system across all services and middlewares, so our measurements are made on a system that is not idle.
We are going to stress the public API endpoint with 2 different requests:
This is interesting because to get the number of steps of a user the public API service needs to make a HTTP request to the activity service, which in turn queries a PostgreSQL database. To fetch a JWT token there is more work to do, as the user profile service need to be queried twice, before doing some cryptography work and finally returning a JWT token. The overall latency for these requests is thus impacted by the work done in the HTTP API, in the user and activity services, and finally in the databases.
The goal here is not to identify the limits of the services in terms of maximum throughput and best latency. We want to have a baseline to see how the service behave under a sustained workload, which will later help us in characterizing the impact of various types of failures and mitigation strategies.
Since Hey cannot be scripted, we have to focus on 1 user. You will find helper scripts in the part2-steps-challenge/load-testing folder. The first script is create-user.sh given in listing 12.3.
#!/bin/sh http http://localhost:4000/api/v1/register \ #1 username="loadtesting-user" \ password="13tm31n" \<EMAIL_ADDRESS>\ deviceId="p-123456-abcdef" \ city="Lyon" \ makePublic:=true for n in `seq 10`; do #2 http http://localhost:3002/ingest \ deviceId="p-123456-abcdef" \ deviceSync:=$n \ stepsCount:=1200 done
The script ensures that user loadtesting-user is created, and that a few updates have been recorded. The run-hey-user-steps.sh script found in listing 12.4 uses Hey and fetches the total number of steps of user loadtesting-user.
 In my case I am doing the experiment with a second laptop, while my main laptop runs the services and had IP address <IP_ADDRESS> when I ran the tests. First off we get some light background workload with Locust, again to make sure the system is not exactly idle:
$ locust --no-web --host http://<IP_ADDRESS> --clients 20 --hatch-rate 2
In another terminal we launch the test with Hey for 5 minutes:
./run-hey-user-steps.sh <IP_ADDRESS> 5m
Once we have collected the results the best way to analyze them is to process the data and plot it. You will find Python scripts to do that in the part2-steps-challenge/load-testing folder. Figure 12.3 shows the plot for the experiment above.
The figure contains 3 sub-plots:
The 99.99th percentile latency is very good while the throughput is high. We get better results with Hey compared to a 100 users workload with Locust. We can see a few short throughput drops correlated with higher latency responses, but there is nothing to worry about in these conditions. These drops could have been caused by various factors, including PostgreSQL, the WiFi network or a JVM garbage collector run. It is easy to get better results with better hardware running Linux, a wired network, some JVM tuning and a properly configured PostgreSQL database server.
We can run another load testing experiment, now by fetching JWT tokens:
./run-hey-token.sh <IP_ADDRESS> 5m
The results are shown in figure 12.4.
Here the results show again a quite consistent behavior, albeit with a higher latency and lower throughput to what the steps count endpoint could achieve. This is easy to explain as there are 2 HTTP requests to the user profile service, then the token has to be generated and signed. The HTTP requests are mostly I/O-bound, while token signing requires CPU-bound work to be done on the event-loop. The results are consistent over the 5 minutes run.
It is safe to conclude that the tested service implementations deliver solid performance under load. You could try to increase the number of workers for Hey and see what happens with bigger workloads (see the -c flag of the hey tool). You could also perform latency measures with increasing request rates (see the -q flag), but note that by default Hey does not do rate limiting, so in the previous runs Hey did the best it could with 50 workers (the default).
Scalability is only half of being reactive, so let us now see how our services behave with the same workloads in presence of failures.
Strictly speaking, chaos engineering is the practice of voluntarily introducing failures in production systems to see how they react to unexpected application, network and infrastructure failures. For instance you can try to shut down a database, take down a service, introduce network delays or even interrupt traffic between networks.
You can also do chaos engineering before software hits production, as the core principle remains the same: run the software with some workload, introduce some form of failure, and see how the software behaves.
We need a reproducible scenario to evaluate the services as they will alternate between nominal and failure phases. We are going to introduce failures according to the plan in figure 12.5.
We are going to run the same load testing experiments as we did over periods of 5 minutes. What is going to change is that we are going to split into 5 phases of 1 minute each:
Network delays increase latency, but they also simulate an overloaded database or service that starts to become unresponsive. With extreme delay values they can also simulate an unreachable host where establishing TCP connections takes a long time to fail. On the other hand stopping the databases simulates services down while their hosts remain up, which should lead to quick TCP connection errors.
So how are we going to introduce these failures?
Pumba is a chaos testing tool for introducing failures in Docker containers [Pumba]. It can be used to:
Pumba is a very convenient tool that you can download and run on your machine. The only dependency is having Docker running.
We are focusing on 2 types of failures in our test plan as they are the most relevant to us. You can play with other types of failures as easily.
With the 10k steps application running locally, let’s play with Pumba and add some delay to the MongoDB database traffic. Let’s fetch a JWT token with the load-testing/fetch-token.sh script, as in listing 12.6.
In another terminal, let’s introduce the delays with the command of listing 12.7.
$ pumba netem \ #1 --duration 1m \ #2 --tc-image gaiadocker/iproute2 \ #3 delay --time 3000 --jitter 500 \ #4 part2-steps-challenge_mongo_1 #5
Pumba should now be running for 1 minute. Try fetching a JWT token again: the command should clearly take more time than before, as shown in listing 12.8.
The process took 6.157 seconds to fetch a token due to waiting for I/O. Similarly, you can stop a container with the command of listing 12.9.
$ pumba stop --restart \ #1 --duration 1m \ part2-steps-challenge_mongo_1
Now when you run the script to fetch a token again, you will be waiting while in the logs you will see some errors due to the MongoDB container to be down, as seen in listing 12.10.
The service is now unresponsive: our request took 57.315 seconds to complete because it had to wait for the database to be back. Let’s have a clearer understanding by running the test plan, and see what happens when these failures happen and the system is under load testing.
To run these experiments you will use the same shell scripts to launch Hey as we did earlier in this chapter. You will preferably use 2 machines. The part2-steps-challenge/load-testing folder contains a run-chaos.sh shell script to automate the test plan by calling Pumba at the right time. The key is to start both the run-chaos.sh and Hey scripts (e.g., run-hey-token.sh) at the same time.
Figure 12.6 shows the behavior of the service on getting a user total steps count.
The results show a clear lack of responsiveness when Pumba runs. In the phase of network delays, we see a rapid latency increase spike to nearly 20 seconds, after which the throughput implodes. What happens here is that requests get enqueued waiting for a response in both the public api and user profile services, up to the point where the system is at a halt. The database delays are between 2.5s and 3.5s, which can temporarily happen in practice. Of course the issue is vastly amplified here due to load testing, but any service with some sustained traffic can show this kind of behavior even with smaller delays.
In the phase of databases being down we see errors for the whole simulated outage duration. While it is hard to be surprised about errors, we can see that the system has not come to a halt either. This is far from perfect though, since the reduced throughput is the sign that requests need some time to be given an error, while other requests are waiting until they time out, or eventually complete when the databases restart.
Let us now look at figure 12.7 and see how fetching JWT tokens goes.
Network delays also cause the system to come to a halt, but we do not observe the same shape in the scatter plot. This is due to the inherently lower throughput of the service for this type of requests, and also the fact that 2 HTTP requests are needed. Requests pile up, waiting for responses to arrive, and once delays stop the system gets going again. More interestingly we do not observe errors in the phase where databases have been stopped, there are just no requests being served anymore as the system is waiting for databases.
From these 2 experiments the services become unresponsive in presence of failures, hence they are not reactive. The good news is that there are ways to fix it, so let’s see how we can become reactive, again using the public API as a reference. You will then be able to extrapolate the techniques to the other services.
To be resilient we have to make changes to the public API and make sure that it responds quickly when a failure has been detected. We are going to explore 2 approaches: enforcing timeouts, then using a circuit breaker.
You can find the corresponding code changes in the chapter12/public-api-with-timeouts branch of the Git repository.
Observations in the experimentations above showed that requests piled up while waiting for either the databases to be back to nominal conditions, or TCP errors to arise. A first approach could be to enforce short timeouts in the HTTP client requests so that they fail fast when the user profile or activity services take tool long to respond. The changes are very simple: we add timeouts to the HTTP requests done by the Vert.x web client, as shown in listing 12.11.
private void totalSteps(RoutingContext ctx) { String deviceId = ctx.user().principal().getString("deviceId"); webClient .get(3001, "localhost", "/" + deviceId + "/total") .timeout(5000) #1 .as(BodyCodec.jsonObject()) .rxSend() .subscribe( resp -> forwardJsonOrStatusCode(ctx, resp), err -> sendBadGateway(ctx, err)); }
The changes are the same in the fetchUserDetails and token methods. A timeout of 5 seconds is relatively short and ensures a quick notification of an error.
Intuitively, this should improve responsiveness of the public API services and avoid throughput coming to a halt. Let us see what happens by running the chaos testing experiments again, starting with figure 12.8.
Compared to the experiment in figure 12.6 we still have drastically reduced throughputs during failures, but at least we see errors being reported thanks to the timeout enforcements. We also see that the maximum latency is below 6 seconds, which is in line with 5 seconds timeouts.
Let us now see how the JWT token load test behaves, as reported in figure 12.9.
This run confirms what we have observed: timeouts get enforced, ensuring that some requests still get served during the failures. However the worst case latencies are worse than without timeouts: network delays stretch the time for doing 2 HTTP requests to the user profile service, so the higher values correspond to those where the second request timed out.
Timeouts are better than no timeouts when it comes to improving responsiveness, but we cannot qualify our public API service as being resilient. What we need is a way for the service to know that there is a failure happening, so it fails fast rather than waiting for a timeout to happen. This is exactly what a circuit breaker is for!
You can find the corresponding code changes in the chapter12/public-api-with-circuit-breaker branch of the Git repository.
The goal of a circuit breaker is to prevent the problems observed above, where requests to unresponsive systems pile-up, causing cascading errors between distributed services. A circuit breaker acts as some form of proxy between the code that makes a (networked) request such as a RPC call, HTTP request or database call, and the service to be invoked. Figure 12.10 shows how a circuit breaker works as a finite state machine.
The idea is quite simple. The circuit breaker starts in state closed, and for each request it observes if the request succeeded or not. Failing can be because an error has been reported (e.g., a TCP timeout, a TCP connection error), or because an operation took too long to complete.
Once a certain amount of errors have been reported, the circuit breaker goes to the open state. From here all operation are notified of a failure due to the circuit being open. This avoids issuing further requests to an unresponsive service, which allows for providing fast error responses, trying alternative recovery strategies, and also releasing the pressure on both the service and requester ends.
The circuit breaker leaves the open state to the half-open state after some reset timeout. The first request in the half-open state is used to know if the service has recovered. Unlike the open state, the half-open state is where we start doing a real operation again. If it succeeds then the circuit breaker goes back to the closed state and resumes normal servicing. If not then another reset period starts before going back to the half-open state and check if the service is back.
Vert.x provides the vertx-circuit-breaker module that needs to be added to the public API project. We are going to use 2 circuit breakers: 1 for token generation requests, and 1 for calls to the activity service (e.g., getting the total steps count of a user). Listing 12.12 shows the code to create a circuit breaker in the PublicApiVerticle rxStart method.
String tokenCircuitBreakerName = "token-circuit-breaker"; tokenCircuitBreaker = CircuitBreaker.create( tokenCircuitBreakerName, vertx, circuitBreakerOptions()); #1 tokenCircuitBreaker .openHandler(v -> this.logBreakerUpdate("open", tokenCircuitBreakerName)); #2 tokenCircuitBreaker .halfOpenHandler(v -> this.logBreakerUpdate("half open", tokenCircuitBreakerName)); #3 tokenCircuitBreaker .closeHandler(v -> this.logBreakerUpdate("closed", tokenCircuitBreakerName)); #4
The tokenCircuitBreakerName reference is a is a field of type CircuitBreaker. There is another field called activityCircuitBreaker for the activity service circuit breaker and the code is identical. The callbacks on state change can be optionally set. It is a good idea to log these state changes for diagnosis purposes. Listing 12.13 shows a circuit breaker configuration.
private CircuitBreakerOptions circuitBreakerOptions() { return new CircuitBreakerOptions() .setMaxFailures(5) #1 .setMaxRetries(0) #2 .setTimeout(5000) #3 .setResetTimeout(10_000); #4 }
We are going to open the circuit breaker after 5 failures, including operations timing out after 5 seconds (to be consistent with the previous experiments). The reset timeout is set to 10 seconds, which lets us frequently check how the service goes. How long shall this value be is up to your context, but you can anticipate that long timeouts increase the time a service operates in degraded mode or reports errors, while short values may diminish the effectiveness of using a circuit breaker.
Listing 12.14 shows the modified token method with the code wrapped into a circuit breaker call.
private void token(RoutingContext ctx) { tokenCircuitBreaker.<String>rxExecute(promise -> { #1 JsonObject payload = ctx.getBodyAsJson(); String username = payload.getString("username"); webClient #2 .post(3000, "localhost", "/authenticate") // (...) #3 .subscribe(promise::complete, err -> { #4 if (err instanceof NoStackTraceThrowable) { #5 promise.complete(""); } else { promise.fail(err); #6 } }); }).subscribe( #7 token -> sendToken(ctx, token), err -> handleAuthError(ctx, err)); }
The circuit breaker executes an operation, which here is making 2 HTTP requests to the user profile service then making a JWT token. The operation result is a Single<String> of the JWT token value. The execution method passes a promise to the wrapped code, so it can notify if the operation succeeded or not. The handleAuthError method had to be modified as in listing 12.15 to check what is the source of any error.
private void handleAuthError(RoutingContext ctx, Throwable err) { if (err instanceof OpenCircuitException) { #1 logger.error("Circuit breaker is closed: {}", tokenCircuitBreaker.name()); ctx.fail(504); } else if (err instanceof TimeoutException) { #2 logger.error("Circuit breaker timeout: {}", tokenCircuitBreaker.name()); ctx.fail(504); } else { #3 logger.error("Authentication error", err); ctx.fail(401); } }
The circuit breaker reports of open circuit conditions and operation timeouts with dedicated exceptions. In these cases we report a HTTP 500 status code, else a classic 401 so the requester knows if a failure is due to bad credentials or not.
Now this is all great, but what is the actual effect of the circuit breaker on our system? Let us see by running the experiment on the JWT token generation and have a look at the results in figure 12.11.
The impact of the circuit breaker is striking: the service is now highly responsive during failure periods! We get a high throughput during failures as the service now fails fast when the circuit breaker is open. Interestingly we can spot when the circuit breaker tries making requests when in half-open state: these are the high-latency error points at regular intervals. We can also see that the 99.99th percentile is back to a lower latency compared to the previous runs.
This is all good, but how about fetching total steps count of a user?
The circuit breaker made JWT token generation responsive even with failures, so the endpoint is now fully reactive. That being said it did not offer much fallback strategies: if we can’t talk to the user profile service then there is no way we can authenticate a user then generate a JWT token. This is why the circuit breaker always reports errors.
We could adopt the same strategy when issuing requests to the activity service, and simply report errors. That being said we can provide further resiliency by caching data and still provide an older value to a requester. Fallback strategies depend on the functional requirements: we cannot generate a JWT token without authentication working, but we can certainly serve some older steps count data if we have it in a cache.
We are going to the use the efficient in-memory Caffeine caching library [Caffeine]. This library provides configurable strategies for managing cached data, including count, access and time-based eviction policies. We could cache data in a Java HashMap, but that would quickly expose us to memory exhaustion problems if we didn’t put a proper eviction policy in place. Listing 12.16 shows us how to create a cache of at most 10 000 entries where keys are strings and values are long integers.
We add entries to the cache with the method cacheTotalSteps of listing 12.17, and Caffeine evicts older entries when the 10 000 entries limit has been reached.
This method is used in the totalSteps method where the code has now been wrapped using a circuit breaker call as in listing 12.18.
private void totalSteps(RoutingContext ctx) { String deviceId = ctx.user().principal().getString("deviceId"); activityCircuitBreaker.<Void>executeWithFallback(promise -> { #1 webClient .get(3001, "localhost", "/" + deviceId + "/total") .expect(ResponsePredicate.SC_OK) .as(BodyCodec.jsonObject()) .rxSend() .subscribe(resp -> { cacheTotalSteps(deviceId, resp); #2 forwardJsonOrStatusCode(ctx, resp); promise.complete(); }, err -> { tryToRecoverFromCache(ctx, deviceId); #3 promise.fail(err); }); }, err -> { #4 tryToRecoverFromCache(ctx, deviceId); return null; }); }
We now use a circuit breaker that does not return any value, hence the Void parametric type. The executeWithFallback method allows to provide a fallback when the circuit is open, so we can try to recover a value from the cache. This is done in method tryToRecoverFromCache of listing 12.19.
private void tryToRecoverFromCache(RoutingContext ctx, String deviceId) { Long steps = stepsCache.getIfPresent("total:" + deviceId); if (steps == null) { #1 logger.error("No cached data for the total steps of device {}", deviceId); ctx.fail(502); } else { #2 JsonObject payload = new JsonObject() .put("count", steps); ctx.response() .putHeader("Content-Type", "application/json") .end(payload.encode()); } }
By doing so we don’t always send errors, if we have data in the cache then we can still provide a response, albeit with a possibly outdated value.
Caching steps count and recovering from older values with a circuit breaker fallback could also be done directly in the activity service.
It is now time to check the behavior of the service when fetching step counts. First, let us have a "cold start" run where the database is initially down and the service has just started. Figure 12.12 shows a 2 minutes run where the database only starts after a minute.
The service immediately starts with a few errors then the circuit breaker opens, at which point the service consistently provides errors with a very low latency. Remember that the service hasn’t cached any data yet. When the database starts we can see a latency spike as errors turn into successes, then the service is able to respond nominally. Note that in the first success seconds the JVM will start optimizing the code that talks to the database, hence an improving throughput. Figure 12.13 shows the service behavior on our full 5 minutes test plan.
Since the test plan starts with databases running nominally, the service manages to cache data for the test user. This is why we get no error across the whole run. We see a few successes with higher latency when the network delays appear, which actually impact the last few percentiles above 9.999th. These are due to the circuit breaker reporting timeouts on making HTTP requests, but note that the circuit breaker cannot cancel the HTTP requests. Hence we have a few HTTP requests waiting for an unresponsive activity service, while the circuit breaker meanwhile completes the corresponding HTTP responses with some cached data. Figure 12.14 shows the effect of combining the circuit breaker with a 5 seconds timeout on the web client HTTP requests (see the chapter12/public-api-with-circuit-breaker-and-timeouts branch of the Git repository).
This clearly improves the result, as we don’t have any worst-case latency around 20 seconds anymore. Other than that the latency and throughputs are consistent over the rest of the run, and barely impacted by the databases being stopped around minute 4.
A circuit breaker is a very useful tool for avoiding cascading failures, but you don’t have to wrap every operation over the network in a circuit breaker. Every abstraction has a cost, and circuit breakers do add a level of indirection. Instead it is best to use chaos testing and identify where they are most likely to have a positive effect on the overall system behavior.
We now have an actually reactive service: it is not just resource-efficient and scalable, but it is also resilient to failures. The service keeps responding in all situations and the latency is kept under control.
The next and final chapter discusses running Vert.x applications in container environments.
[Locust] Locust project. Locust. Retrieved April 2020. locust.io/
[Hey] Hey project. Hey. Retrieved April 2020. github.com/rakyll/hey
[COP] Gil Tene. How NOT to Measure Latency. 2015. Talk given at the Strange Loop conference. www.youtube.com/watch?v=lJ8ydIuPFeU
[Pumba] Pumba project. Pumba. Retrieved April 2020. github.com/alexei-led/pumba
[CircuitBreaker] Martin Fowler. CircuitBreaker. March 2014. martinfowler.com/bliki/CircuitBreaker.html
[Caffeine] Ben Manes and contributors. Caffeine. Retrieved April 2020. github.com/ben-manes/caffeine


?Manilkara
C^J/
Ur^
B
New
York
Botanical
Garden
Arthur
Cronquist,
1945
Drt.
O.
u,
vÑ*
.pf/e)
D#t
O*
£|.
ffllly
/'
r*<^
.
THE
NEW
YORK
BOTANICAL
GARDEN
EXPLORATION
OF
CUBA
VICINITY
OF
LA
GLORIA.
CAMAQUEY
NO-3-.6L0
"l-YT^
%
5
J.
A.
SHAFER,
COLLECTOR
FEBRUAH7
-,
1909
01442841
Legend -
Level of confidence that token is an accurately-transcribed word
extremely low
very low
low
undetermined
medium
high
very high


Class: Aws::AutoScalingPlans::Types::ScalingPlanResource
| Relationships & Source Files | |
| Super Chains via Extension / Inclusion / Inheritance | |
|
Class Chain:
self,
Struct
|
|
|
Instance Chain:
self,
::Aws::Structure ,
Struct
|
|
| Inherits: |
Struct
|
| Defined in: | gems/aws-sdk-autoscalingplans/lib/aws-sdk-autoscalingplans/types.rb |
Overview
Represents a scalable resource.
Constant Summary
Instance Attribute Summary
-
#resource_id ⇒ String
rw
The ID of the resource.
-
#scalable_dimension ⇒ String
rw
The scalable dimension for the resource.
-
#scaling_plan_name ⇒ String
rw
The name of the scaling plan.
-
#scaling_plan_version ⇒ Integer
rw
The version number of the scaling plan.
-
#scaling_policies ⇒ Array<Types::ScalingPolicy>
rw
The scaling policies.
-
#scaling_status_code ⇒ String
rw
The scaling status of the resource.
-
#scaling_status_message ⇒ String
rw
A simple message about the current scaling status of the resource.
-
#service_namespace ⇒ String
rw
The namespace of the AWS service.
Instance Method Summary
::Aws::Structure
- Included
| #to_hash | Alias for Structure#to_h. |
Instance Attribute Details
#resource_id ⇒ String
(rw)
The ID of the resource. This string consists of the resource type and unique identifier.
Auto Scaling group - The resource type is
autoScalingGroup
and the unique identifier is the name of the Auto Scaling group. Example:autoScalingGroup/my-asg
.::Aws::ECS
service - The resource type isservice
and the unique identifier is the cluster name and service name. Example:service/default/sample-webapp
.Spot Fleet request - The resource type is
spot-fleet-request
and the unique identifier is the Spot Fleet request ID. Example:spot-fleet-request/sfr-73fbd2ce-aa30-494c-8788-1cee4EXAMPLE
.::Aws::DynamoDB
table - The resource type istable
and the unique identifier is the resource ID. Example:table/my-table
.::Aws::DynamoDB
global secondary index - The resource type isindex
and the unique identifier is the resource ID. Example:table/my-table/index/my-table-index
.Aurora DB cluster - The resource type is
cluster
and the unique identifier is the cluster name. Example:cluster:my-db-cluster
.
# File 'gems/aws-sdk-autoscalingplans/lib/aws-sdk-autoscalingplans/types.rb', line 1032
class ScalingPlanResource < Struct.new( :scaling_plan_name, :scaling_plan_version, :service_namespace, :resource_id, :scalable_dimension, :scaling_policies, :scaling_status_code, : ) SENSITIVE = [] include Aws::Structure end
#scalable_dimension ⇒ String
(rw)
The scalable dimension for the resource.
autoscaling:autoScalingGroup:DesiredCapacity
- The desired capacity of an Auto Scaling group.ecs:service:DesiredCount
- The desired task count of an::Aws::ECS
service.ec2:spot-fleet-request:TargetCapacity
- The target capacity of a Spot Fleet request.dynamodb:table:ReadCapacityUnits
- The provisioned read capacity for a DynamoDB table.dynamodb:table:WriteCapacityUnits
- The provisioned write capacity for a DynamoDB table.dynamodb:index:ReadCapacityUnits
- The provisioned read capacity for a DynamoDB global secondary index.dynamodb:index:WriteCapacityUnits
- The provisioned write capacity for a DynamoDB global secondary index.rds:cluster:ReadReplicaCount
- The count of Aurora Replicas in an Aurora DB cluster. Available for Aurora MySQL-compatible edition and Aurora PostgreSQL-compatible edition.
# File 'gems/aws-sdk-autoscalingplans/lib/aws-sdk-autoscalingplans/types.rb', line 1032
class ScalingPlanResource < Struct.new( :scaling_plan_name, :scaling_plan_version, :service_namespace, :resource_id, :scalable_dimension, :scaling_policies, :scaling_status_code, : ) SENSITIVE = [] include Aws::Structure end
#scaling_plan_name ⇒ String
(rw)
The name of the scaling plan.
# File 'gems/aws-sdk-autoscalingplans/lib/aws-sdk-autoscalingplans/types.rb', line 1032
class ScalingPlanResource < Struct.new( :scaling_plan_name, :scaling_plan_version, :service_namespace, :resource_id, :scalable_dimension, :scaling_policies, :scaling_status_code, : ) SENSITIVE = [] include Aws::Structure end
#scaling_plan_version ⇒ Integer
(rw)
The version number of the scaling plan.
# File 'gems/aws-sdk-autoscalingplans/lib/aws-sdk-autoscalingplans/types.rb', line 1032
class ScalingPlanResource < Struct.new( :scaling_plan_name, :scaling_plan_version, :service_namespace, :resource_id, :scalable_dimension, :scaling_policies, :scaling_status_code, : ) SENSITIVE = [] include Aws::Structure end
#scaling_policies ⇒ Array
<Types::ScalingPolicy> (rw)
The scaling policies.
# File 'gems/aws-sdk-autoscalingplans/lib/aws-sdk-autoscalingplans/types.rb', line 1032
class ScalingPlanResource < Struct.new( :scaling_plan_name, :scaling_plan_version, :service_namespace, :resource_id, :scalable_dimension, :scaling_policies, :scaling_status_code, : ) SENSITIVE = [] include Aws::Structure end
#scaling_status_code ⇒ String
(rw)
The scaling status of the resource.
Active
- The scaling configuration is active.Inactive
- The scaling configuration is not active because the scaling plan is being created or the scaling configuration could not be applied. Check the status message for more information.PartiallyActive
- The scaling configuration is partially active because the scaling plan is being created or deleted or the scaling configuration could not be fully applied. Check the status message for more information.
# File 'gems/aws-sdk-autoscalingplans/lib/aws-sdk-autoscalingplans/types.rb', line 1032
class ScalingPlanResource < Struct.new( :scaling_plan_name, :scaling_plan_version, :service_namespace, :resource_id, :scalable_dimension, :scaling_policies, :scaling_status_code, : ) SENSITIVE = [] include Aws::Structure end
#scaling_status_message ⇒ String
(rw)
A simple message about the current scaling status of the resource.
# File 'gems/aws-sdk-autoscalingplans/lib/aws-sdk-autoscalingplans/types.rb', line 1032
class ScalingPlanResource < Struct.new( :scaling_plan_name, :scaling_plan_version, :service_namespace, :resource_id, :scalable_dimension, :scaling_policies, :scaling_status_code, : ) SENSITIVE = [] include Aws::Structure end
#service_namespace ⇒ String
(rw)
The namespace of the AWS service.
# File 'gems/aws-sdk-autoscalingplans/lib/aws-sdk-autoscalingplans/types.rb', line 1032
class ScalingPlanResource < Struct.new( :scaling_plan_name, :scaling_plan_version, :service_namespace, :resource_id, :scalable_dimension, :scaling_policies, :scaling_status_code, : ) SENSITIVE = [] include Aws::Structure end


FreeCodeCamp Survey Form
Lorem Ipsum Lorem Ipsum is simply dummy text of the printing and typesetting industry?
Name:
Email
Student Number
Choose a day:
Monday
Tuesday
Wednesday
Thursday
Friday
Name
Email
Student Number
Car
Bus
Train
Submit


How to Beat Covid?
The best way till now to fight Covid-19 is to avoid being exposed to the virus
What is Covid-19?
COVID-19 is an infection caused by the new coronavirus that is reported to be emerged in Wuhan, China in December 2019
The new coronavirus can be spread from person to person. It is diagnosed with a laboratory test
There is no coronavirus vaccine yet. The only way to deal with this disease for now is prevention
A lot of countries have gone into repetitive lockdowns to deal with this virus


Let's see, where were we?
Oh, yes. We're in Hollywood, on the set of "Stab 3," an another sequel in the saga of the Woodsboro murders.
Before that, though, we are on a Los Angeles freeway where a boyhood friend of slasher victim Sidney Prescott (Neve Campbell) gets a strange phone call.
Rushing home he enters his apartment and ...
Welcome to Wes Craven's "Scream 3." And like the first two movies in the series, this one is mixes an awareness of the slasher movie genre with audience expectations. But by now the formula is a bit stale, so the audience is - if not ahead - at least able to keep pace with Craven and screenwriter Ehren Kruger.
OK, so back to the lot of Sunrise Pictures, where a replica of Sidney's house stands on a closed set and a cast of two-dimensional types sit around complaining about the script and their parts.
Of course, now the cast members begin to get bumped off so ...
Enter the famous newswoman Gale Weathers (Courtney Cox Arquette, as she now bills herself) to get the scoop. And who better, since she was a player in alll the previous action and even wrote a book about it.
Also on hand is Dewey (David Arquette), the film's technical adviser and Gale's former boyfriend, who still carries a torch.
Ain't love grand. What better to reunite two former lovers than a good serial killing spree.
As for Sidney, she is holed up somewhere with enough security locks to keep out the CIA. But the ghost-masked killer learns where she is, so she flees her isolation, heads for L.A., hooks up with Gale and Dewey and we're off to the races.
At about 1 hour and 50 minutes, "Scream 3" drags. There is too much exposition, the pace is uneven and the rationalization for everything two complex.
But what the heck. The movie will make money because those who were fascinated by "Scream" and "Scream 2" won't be able to resist this outing.
Craven offers some good scary moments and a few goosebumps, but it seems so formulated, so rote. It lacks a freshness. The movie has a deja vu, haven't we been here before, feel.
The cast just goes through its motions, no one stands out and no one does anything special. "Scream 3" is just more of the same old, same old
Even cameos by Roger Corman, Carrie Fisher and Kevin Smith as Silent Bob and Jason Mewes as his motor-mouthed pal from "Dogma" cannot enliven the plodding proceedings.
It is time to retire this worn out series. Come on, Wes. Put the knives and ghost masks away and keep them locked up. Move on. "Scream 3" is like the party guest who stayed too long and wore out his welcome.
Bob Bloom is the film critic at the Journal and Courier in Lafayette, IN. He can be reached by e-mail at<EMAIL_ADDRESS>or at<EMAIL_ADDRESS>The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


Since I am only one person running this, these are the times during the day that I can submit tweaks to the repo:
Monday - Friday >>> 5:00 PM - 7:00 PM UTC -4 (EDT)
Saturday - Sunday >>> 1:00 PM - 8:00 PM UTC -4 (EDT)
You don't have to worry about this - just email me with your tweak stuff and I'll put it up whenever I can. If something goes wrong, I will contact you.
Your tweak should support at least iOS 10 and above.
I will not host any pirating tweak or anything related to that. Thanks for understanding.
To add your tweak to the repo, please email me at<EMAIL_ADDRESS>with your tweak deb file and a short description of the tweak (to make sure it is not harmful). I also need proof that the tweak is of your own / you got permission from a tweak dev to host the tweak here. Your control file should have the description you want, as I don't currently have depictions.


Diskografie
The Sacrament Of Sin
(2018)
- Fire & Forgive
- Demons Are A Girl‘s Best Friend
- Killers With The Cross
- Incense And Iron
- Where The Wild Wolves Have Gone
- Stossgebet
- Nightside Of Siberia
- The Sacrament Of Sin
- Venom Of Venus
- Nighttime Rebel
- Fist By Fist (Sacralize Or Strike)
- Midnight Madonna (Bonus Track – nur im EMP- und Napalm-Records-Mailorder)
Texty
písniček
Blessed & Possessed
(2015)
- Blessed & Possessed
- Dead Until Dark
- Army Of The Night
- Armata Strigoi
- We Are The Wild
- Higher Than Heaven
- Christ & Combat
- Sanctus Dominus
- Sacramental Sister
- All You Can Bleed
- Let There be Night
Texty
písniček
Preacher Of The Night
(2013)
- Amen & Attack
- Secrets of the sacristy
- Coleus sanctus
- Sacred & wild
- Kreuzfeuer
- Cardinal Sin
- In the name of god
- Nochnoi dozor
- Lust for blood
- Extatum et oratum
- Last of the living dead
Texty
písniček
Blood Of The Saints
(2011)
- Agnus dei (Intro)
- Sanctified with dynamite
- We drink your blood
- Murder at Midnight
- All we need is blood
- Dead boys don’t cry
- Son of a wolf
- Night of the werewolves
- Phantom of the funeral
- Die, die, crucified
- Ira Sancti (When the saints are going wild)
Texty
písniček
Bible Of The Beast
(2009)
- Prelude to purgatory (Intro)
- Raise your fist, Evangelist
- Moscow after dark
- Panic in the pentagram
- Catholic in the morning… Satanist at night
- Seven deadly saints
- Werewolves of Armenia
- We take the church by storm
- Resurrection by erection
- Midnight messiah
- St. Satans day
- Wolves against the world
Texty
písniček
- We take it from the livinge
- Prayer in the dark
- Moscow after dark
- Saturday satan
- In blood we trust
- Behind the leathermask
- When the moon shines red
- Mother Mary is a bird of prey
- Tiger of Sabrod
- Lupus dei
Texty
písniček
Return in Bloodred
(2005)
- Mr. Sinister
- Kiss of the cobra king
- We came to take your souls
- Black mass hysteria
- Demons & Diamonds
- Montecore
- The evil made me do it
- Lucifer in starlight
- Son of the morning star
Texty
písniček


Sponsor Us
Why Help the PyLadies?
PyLadies want to bring fresh talent into the Python community and to support both seasoned and budding developers through workshops, community activities, and local advocacy. Your support of PyLadies' outreach efforts will pay off hugely over the long term by improving the diversity of the software industry as a whole.
How you can help
- Email us, indicating how you would like to sponsor Pyladies.
- Make a donation through the Python Software Foundation. Donations made are tax-deductible. If you would prefer not to use credit card or PayPal, please contact us to make other arrangements.
If you know of a talented, deserving developer who might benefit greatly from attending a Python or Open Source conference, let us know. We'll do our best to help as many women as possible.


This guide explains the conventions used by utopia-project
when generating documentation for your project.
Source Code Documentation
Source code documentation is expected to be in markdown format. This is different from the canonical RDoc
format used by Ruby. However, using markdown is a good format for standardised documentation across a range of different languages.
Tags
@param
The @param
tag is used to describe the parameters of a method:
# @param x [Integer] The x co-ordinate.
# @param y [Integer] The y co-ordinate.
def move(x, y)
# ...
end
@return
The @return
tag is used to describe the return type of the definition:
# @return [Integer] The result of the computation.
def fib(n)
# ...
end
References
It is possible to insert references in your documentation using curly brackets. In the following example {Input}
will be expanded relative to the usage, to some symbol named Input
.
# Frobulates the input, see {Input} for more details.
def frobulate(input)
# ... left to the imagination ...
end
Examples & Guides
Examples provide structured information to help users understand your project and are located in the examples/
directory. One sub-directory per example.
README.md
If an example has a README.md
file, it is used as the main source of documentation.
Source Files
All other source files are listed on the example page. Top level comments with trailing code (segments) are used to help guide the user through the example.


{% macro mention_notification(other) %}
{{ other.real_name|e if other.real_name else other.name|e }}
{{ " " }} has mentioned you in their post!
{% endmacro %}
{% macro reply_post_notification(other) %}
{{ other.real_name|e if other.real_name else other.name|e }}
{{ " " }} has replied to one of your images!
{% endmacro %}
{% macro reply_image_notification(other) %}
{{ other.real_name|e if other.real_name else other.name|e }}
{{ " " }} has replied to one of your posts!
{% endmacro %}
{% macro shared_post_notification(other) %}
{{ other.real_name|e if other.real_name else other.name|e }}
{{ " " }} has shared one of you images!
{% endmacro %}
{% macro shared_image_notification(other) %}
{{ other.real_name|e if other.real_name else other.name|e }}
{{ " " }} has shared one of you posts!
{% endmacro %}
{% macro friend_notification(other) %}
{{ other.real_name|e if other.real_name else other.name|e }}
{{ " " }} is now a friend of yours! Start the talk?
{% endmacro %}
{% macro follower_notification(other) %}
{{ other.real_name|e if other.real_name else other.name|e }}
{{ " " }} now follows you! There gotta be something you want to share.
{% endmacro %}


new cool form
thanks you for doing this survey
Name
Email
Age
status
Unemployed
Employed
Student
Male
Female
Progaming language
Java
C++
Python
something


@include("./../blocks/header/header.html")
@include("./../blocks/roadmap/roadmap.html")
@include("./../blocks/advantages/advantages.html")
@include("./../blocks/knowledge/knowledge.html")
@include("./../blocks/security/security.html")
@include("./../blocks/promo/promo.html")
@include("./../blocks/skills/skills.html")
@include("./../blocks/program/program.html")
@include("./../blocks/instructor/instructor.html")
@include("./../blocks/promo/promo.html")
@include("./../blocks/reviews/reviews.html")
@include("./../blocks/footer/footer.html")


--- layout: archive title: Zulip Chat Archive permalink: /stream/113488-general/topic/extensionality.20as.20a.20typeclass.html ---
For all these bundled homs, the extensionality lemma can be expressed as saying that ⇑
is injective. Is this some well-known type theory concept? Is injective coe
a thing?
Is this something to do with norm_cast
?
Isn't this basically the condition behind a concrete category?
{% endraw %}Last updated: Apr 24 2020 at 19:02 UTC


# Copyright 2019 The Feast Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import pandas as pd
from typing import List, Optional
from collections import OrderedDict
from typing import Dict
from feast.source import Source
from pandas.api.types import is_datetime64_ns_dtype
from feast.entity import Entity
from feast.feature import Feature, Field
from feast.core.FeatureSet_pb2 import FeatureSetSpec as FeatureSetSpecProto
from google.protobuf.duration_pb2 import Duration
from feast.type_map import python_type_to_feast_value_type
from google.protobuf.json_format import MessageToJson
from google.protobuf import json_format
from feast.type_map import DATETIME_COLUMN
from feast.loaders import yaml as feast_yaml
[docs]class FeatureSet:
"""
Represents a collection of features and associated metadata.
"""
def __init__(
self,
name: str,
features: List[Feature] = None,
entities: List[Entity] = None,
source: Source = None,
max_age: Optional[Duration] = None,
):
self._name = name
self._fields = OrderedDict() # type: Dict[str, Field]
if features is not None:
self.features = features
if entities is not None:
self.entities = entities
if source is None:
self._source = None
else:
self._source = source
self._max_age = max_age
self._version = None
self._client = None
self._busy_ingesting = False
self._is_dirty = True
def __eq__(self, other):
if not isinstance(other, FeatureSet):
return NotImplemented
for key in self.fields.keys():
if key not in other.fields.keys() or self.fields[key] != other.fields[key]:
return False
if self.name != other.name or self.max_age != other.max_age:
return False
return True
def __str__(self):
return str(MessageToJson(self.to_proto()))
@property
def fields(self) -> Dict[str, Field]:
"""
Returns a dict of fields from this feature set
"""
return self._fields
@property
def features(self) -> List[Feature]:
"""
Returns a list of features from this feature set
"""
return [field for field in self._fields.values() if isinstance(field, Feature)]
@features.setter
def features(self, features: List[Feature]):
"""
Sets the active features within this feature set
Args:
features: List of feature objects
"""
for feature in features:
if not isinstance(feature, Feature):
raise Exception("object type is not a Feature: " + str(type(feature)))
for key in list(self._fields.keys()):
if isinstance(self._fields[key], Feature):
del self._fields[key]
if features is not None:
self._add_fields(features)
@property
def entities(self) -> List[Entity]:
"""
Returns list of entities from this feature set
"""
return [field for field in self._fields.values() if isinstance(field, Entity)]
@entities.setter
def entities(self, entities: List[Entity]):
"""
Sets the active entities within this feature set
Args:
entities: List of entities objects
"""
for entity in entities:
if not isinstance(entity, Entity):
raise Exception("object type is not na Entity: " + str(type(entity)))
for key in list(self._fields.keys()):
if isinstance(self._fields[key], Entity):
del self._fields[key]
if entities is not None:
self._add_fields(entities)
@property
def name(self):
"""
Returns the name of this feature set
"""
return self._name
@name.setter
def name(self, name):
"""
Sets the name of this feature set
"""
self._name = name
@property
def source(self):
"""
Returns the source of this feature set
"""
return self._source
@source.setter
def source(self, source: Source):
"""
Sets the source of this feature set
"""
self._source = source
@property
def version(self):
"""
Returns the version of this feature set
"""
return self._version
@version.setter
def version(self, version):
"""
Sets the version of this feature set
"""
self._version = version
@property
def max_age(self):
"""
Returns the maximum age of this feature set. This is the total maximum
amount of staleness that will be allowed during feature retrieval for
each specific feature row that is looked up.
"""
return self._max_age
@max_age.setter
def max_age(self, max_age):
"""
Set the maximum age for this feature set
"""
self._max_age = max_age
[docs] def add(self, resource):
"""
Adds a resource (Feature, Entity) to this Feature Set.
Does not register the updated Feature Set with Feast Core
Args:
resource: A resource can be either a Feature or an Entity object
"""
if resource.name in self._fields.keys():
raise ValueError(
'could not add field "'
+ resource.name
+ '" since it already exists in feature set "'
+ self._name
+ '"'
)
if issubclass(type(resource), Field):
return self._set_field(resource)
raise ValueError("Could not identify the resource being added")
def _set_field(self, field: Field):
self._fields[field.name] = field
return
[docs] def drop(self, name: str):
"""
Removes a Feature or Entity from a Feature Set. This does not apply
any changes to Feast Core until the apply() method is called.
Args:
name: Name of Feature or Entity to be removed
"""
if name not in self._fields:
raise ValueError("Could not find field " + name + ", no action taken")
if name in self._fields:
del self._fields[name]
return
def _add_fields(self, fields: List[Field]):
"""
Adds multiple Fields to a Feature Set
Args:
fields: List of Field (Feature or Entity) Objects
"""
for field in fields:
self.add(field)
[docs] def infer_fields_from_df(
self,
df: pd.DataFrame,
entities: Optional[List[Entity]] = None,
features: Optional[List[Feature]] = None,
replace_existing_features: bool = False,
replace_existing_entities: bool = False,
discard_unused_fields: bool = False,
rows_to_sample: int = 100,
):
"""
Adds fields (Features or Entities) to a feature set based on the schema
of a Datatframe. Only Pandas dataframes are supported. All columns are
detected as features, so setting at least one entity manually is
advised.
Args:
df: Pandas dataframe to read schema from
entities: List of entities that will be set manually and not
inferred. These will take precedence over any existing entities
or entities found in the dataframe.
features: List of features that will be set manually and not
inferred. These will take precedence over any existing feature
or features found in the dataframe.
replace_existing_features: If true, will replace
existing features in this feature set with features found in
dataframe. If false, will skip conflicting features.
replace_existing_entities: If true, will replace existing entities
in this feature set with features found in dataframe. If false,
will skip conflicting entities.
discard_unused_fields: Boolean flag. Setting this to True will
discard any existing fields that are not found in the dataset or
provided by the user
rows_to_sample: Number of rows to sample to infer types. All rows
must have consistent types, even values within list types must
be homogeneous
"""
if entities is None:
entities = list()
if features is None:
features = list()
# Validate whether the datetime column exists with the right name
if DATETIME_COLUMN not in df:
raise Exception("No column 'datetime'")
# Validate the data type for the datetime column
if not is_datetime64_ns_dtype(df.dtypes[DATETIME_COLUMN]):
raise Exception(
"Column 'datetime' does not have the correct type: datetime64[ns]"
)
# Create dictionary of fields that will not be inferred (manually set)
provided_fields = OrderedDict()
for field in entities + features:
if not isinstance(field, Field):
raise Exception(f"Invalid field object type provided {type(field)}")
if field.name not in provided_fields:
provided_fields[field.name] = field
else:
raise Exception(f"Duplicate field name detected {field.name}.")
new_fields = self._fields.copy()
output_log = ""
# Add in provided fields
for name, field in provided_fields.items():
if name in new_fields.keys():
upsert_message = "created"
else:
upsert_message = "updated (replacing an existing field)"
output_log += (
f"{type(field).__name__} {field.name}"
f"({field.dtype}) manually {upsert_message}.\n"
)
new_fields[name] = field
# Iterate over all of the columns and create features
for column in df.columns:
column = column.strip()
# Skip datetime column
if DATETIME_COLUMN in column:
continue
# Skip user provided fields
if column in provided_fields.keys():
continue
# Only overwrite conflicting fields if replacement is allowed
if column in new_fields:
if (
isinstance(self._fields[column], Feature)
and not replace_existing_features
):
continue
if (
isinstance(self._fields[column], Entity)
and not replace_existing_entities
):
continue
# Store this field as a feature
new_fields[column] = Feature(
name=column,
dtype=_infer_pd_column_type(column, df[column], rows_to_sample),
)
output_log += f"{type(new_fields[column]).__name__} {new_fields[column].name} ({new_fields[column].dtype}) added from dataframe.\n "
# Discard unused fields from feature set
if discard_unused_fields:
keys_to_remove = []
for key in new_fields.keys():
if not (key in df.columns or key in provided_fields.keys()):
output_log += f"{type(new_fields[key]).__name__} {new_fields[key].name} ({new_fields[key].dtype}) removed because it is unused.\n"
keys_to_remove.append(key)
for key in keys_to_remove:
del new_fields[key]
# Update feature set
self._fields = new_fields
print(output_log)
def _update_from_feature_set(self, feature_set, is_dirty: bool = True):
"""
Deep updates one feature set with another
Args:
feature_set: Feature set to use as a source of configuration
is_dirty: If true, will set this feature set to "dirty" after
this update. This is used to track whether the feature
set needs to be reapplied.
"""
self.name = feature_set.name
self.version = feature_set.version
self.source = feature_set.source
self.max_age = feature_set.max_age
self.features = feature_set.features
self.entities = feature_set.entities
self._is_dirty = is_dirty
[docs] def get_kafka_source_brokers(self) -> str:
"""
Get the broker list for the source in this feature set
"""
if self.source and self.source.source_type is "Kafka":
return self.source.brokers
raise Exception("Source type could not be identified")
[docs] def get_kafka_source_topic(self) -> str:
"""
Get the topic that this feature set has been configured to use as source
"""
if self.source and self.source.source_type == "Kafka":
return self.source.topic
raise Exception("Source type could not be identified")
[docs] def is_valid(self):
"""
Validates the state of a feature set locally
:return: (bool, str) True if valid, false if invalid. Contains a message
string with a reason
"""
if len(self.entities) == 0:
return False, f"No entities found in feature set {self.name}"
return True, ""
[docs] @classmethod
def from_yaml(cls, yml: str):
"""
Creates a feature set from a YAML string body or a file path
Args:
yml: Either a file path containing a yaml file or a YAML string
Returns:
Returns a FeatureSet object based on the YAML file
"""
return cls.from_dict(feast_yaml.yaml_loader(yml, load_single=True))
[docs] @classmethod
def from_dict(cls, fs_dict):
"""
Creates a feature set from a dict
Args:
fs_dict: A dict representation of a feature set
Returns:
Returns a FeatureSet object based on the feature set dict
"""
if ("kind" not in fs_dict) and (fs_dict["kind"].strip() != "feature_set"):
raise Exception(f"Resource kind is not a feature set {str(fs_dict)}")
feature_set_proto = json_format.ParseDict(
fs_dict, FeatureSetSpecProto(), ignore_unknown_fields=True
)
return cls.from_proto(feature_set_proto)
[docs] @classmethod
def from_proto(cls, feature_set_proto: FeatureSetSpecProto):
"""
Creates a feature set from a protobuf representation of a feature set
Args:
from_proto: A protobuf representation of a feature set
Returns:
Returns a FeatureSet object based on the feature set protobuf
"""
feature_set = cls(
name=feature_set_proto.name,
features=[
Feature.from_proto(feature) for feature in feature_set_proto.features
],
entities=[
Entity.from_proto(entity) for entity in feature_set_proto.entities
],
max_age=feature_set_proto.max_age,
source=(
None
if feature_set_proto.source.type == 0
else Source.from_proto(feature_set_proto.source)
),
)
feature_set._version = feature_set_proto.version
feature_set._is_dirty = False
return feature_set
[docs] def to_proto(self) -> FeatureSetSpecProto:
"""
Converts a feature set object to its protobuf representation
Returns:
FeatureSetSpec protobuf
"""
return FeatureSetSpecProto(
name=self.name,
version=self.version,
max_age=self.max_age,
source=self.source.to_proto() if self.source is not None else None,
features=[
field.to_proto()
for field in self._fields.values()
if type(field) == Feature
],
entities=[
field.to_proto()
for field in self._fields.values()
if type(field) == Entity
],
)
def _infer_pd_column_type(column, series, rows_to_sample):
dtype = None
sample_count = 0
# Loop over all rows for this column to infer types
for key, value in series.iteritems():
sample_count += 1
# Stop sampling at the row limit
if sample_count > rows_to_sample:
continue
# Infer the specific type for this row
current_dtype = python_type_to_feast_value_type(name=column, value=value)
# Make sure the type is consistent for column
if dtype:
if dtype != current_dtype:
raise ValueError(
f"Type mismatch detected in column {column}. Both "
f"the types {current_dtype} and {dtype} "
f"have been found."
)
else:
# Store dtype in field to type map if it isnt already
dtype = current_dtype
return dtype


Try to answer the following FRIENDS questions within the time limit. Keep in mind that incorrect answers will penalize your score/time by ten seconds!
Please enter your initials.


Links to resources
In swift succession, we received in 2006 two stemmers for Romanian
written in Snowball.
Here is the original correspondence,
FROM Erwin Glockner<EMAIL_ADDRESS>TO<EMAIL_ADDRESS>ON Wed Jun 07 08:56:44 2006
[Snowball-discuss] romanian stemmer
Hello everyone,
my name is Erwín Glockner, I'm a student of computational linguistics in
Heidelberg, Germany. Together with my fellow students Doina Gliga and
Marina Stegarescu we started to write a romanian stemmer in Snowball.
We planned to finish the stemmer until end of this month. We would be
happy if the stemmer would be accepted as part of the Snowball-distribution.
There is still some work to do, e.g. evaluating the stemmer, making a
stopwords-list, unicode support, etc. After finishing this we will send
you our stemmer with the corresponding files, but I couldn't find any
email adress to whom the stemmer should be sent to.
Could please someone tell me the address(es)?
With kind regards,
E. Glockner, D. Gliga, M. Stegarescu.
FROM Erwin Glockner<EMAIL_ADDRESS>TO<EMAIL_ADDRESS>TO<EMAIL_ADDRESS>ON Tue Jul 18 19:43:39 2006
romanian stemmer
Dear Mr. Porter, dear Mr. Boulton,
we finally finished the Romanian stemmer. Unfortunately evaluation took
more time than expected.
However, it was an interesting experience creating the stemmer, and we
are happy to send you the result of our work.
The attachment-file is a Tarball-zipped file with (hopefully) all files
needed. The files and the stemmer as well are encoded in UTF-8. Please
inform us if something is missing.
We would be happy if the Romanian stemmer would be accepted and
integrated into the official Snowball distribution. We agree of course
to license the stemmer under the same terms as the existing snowball
software.
We're looking forward to hear from you soon.
With kind regards,
Marina, Doina and Erwin.
Attachment: [romanian1.tgz]
FROM Irina Tirdea<EMAIL_ADDRESS>TO<EMAIL_ADDRESS>TO<EMAIL_ADDRESS>TO<EMAIL_ADDRESS>ON Mon Jul 31 10:19:51 2006
Romanian stemmer
Hello,
My name is Irina Tirdea and I have developed a Romanian stemmer in Snowball
as part of my bachelor thesis, in Bucharest, Romania. I am sending you the
code attached (with vocabulary and stop word list files) and I hope you will
accept and integrate it as a part of the Snowball project. I am ready to
release the stemmer under the BSD license, just as the Snowball software.
The files have been written in UTF-8 encoding (on a Linux system).
Looking forward to hear from you.
Kind regards,
Irina Tirdea
Attachment; [romanian2.tgz]
FROM<EMAIL_ADDRESS>(Martin Porter)
TO<EMAIL_ADDRESS>ON Mon Jul 31 10:43:00 2006
COPIED TO<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Tardy response to submissions to Snowball
I am sending this general email as a kind of apology, for having done nothing
so far on the following generously sent Snowball submissions:
7 June, from E. Glockner: a Romanian stemmer
8 June, from A. Tordai: a Hungarian stemmer
and this morning another Romanian stemmer arrived,
31 July, from I. Tirdea, a Romanian stemmer
After the first submission I promised to look at it "next week", so Mr Glockner
has probably been wondering what has happened. [. . .] I will make a point of
looking at these submissions this week,
More soon,
Martin
FROM<EMAIL_ADDRESS>(Martin Porter)
TO<EMAIL_ADDRESS>ON Wed Sep 06 12:39:13 2006
COPIED TO<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Romanian stemmer
To the originators of the Romanian stemmers,
I have now found time to do some preliminary work on the Romanian stemmer. I
should explain that part of the complication has been the receipt, no more
than ten days apart, of two Romanian stemmers in Snowball, the first
(romanian1) from [Glockner, Gliga, and Stegarescu] in Heidelberg, the second
(romanian2) from Tirdea in Bucharest.
[. . . .]
I have put together a vocabulary by combining the vocabularies provided with
romanian1 and romanian2. This appears in column 1. Column 2 is the stemmed
form produced by romanian1, and column 3 the stemmed form produced by
romanian2. If the entry in column 3 is blank, both stemmers are producing the
same result.
You might care to compare the two approaches.
My own feeling is that romanian1 does a more thorough job of ending removal,
but unlike romanian2 has a habit of discarding too much from short words.
aberant->ab, abatere->ab, aburi->ab are examples of this. In romanian1 the R2
test is rarely used (it seems to me that 'R1 or R2' is equivalent to 'R1',
since p2 is never to the left of p1.)
I might have a go at making some modifications here. Needless to say, I am
not familiar with Romanian, but the similarity to the other Romance
languages, especially Italian, enables one to grasp the essential features of
the morphology.
What we would like to do is to have a single stemmer for release from the
snowball site, if that is possible, and giving all necessary credits, along
the lines of the recent addition,
http://snowball.tartarus.org/algorithms/hungarian/stemmer.html
Hope to hear from you,
Martin Porter
Finally we decided to produce our own Romanian stemmer as described on the
Romanian stemmer page. The submitted stemmers both contain stop word lists,
available inside the tarballs.
|


Navigation inside an iframe toward external software can be seen by users as a
new popup or a new top-level navigation. That's why its is allowed in sandboxed
iframe only when one of allow-popups, allow-top-navigation, allow-top-navigation-by-user-activation,
or allow-top-navigation-to-custom-protocols
is specified.


To view this video please enable JavaScript, and consider upgrading to a web browser that supports HTML5 video


DSC KGEC is a student led club, built on the power of communities. We believe that great things happen when young inquisitive minds come together, to explore new technologies and create stuff. It is also a way for students to bridge the industry-academia gap through peer-based, project focused learning.
Join usDevelopers Student Club KGEC is inspired by the Google Developers' Family. The motive is to create a local ecosystem of Developers in and around the Campus. And having fun doing it.
 At DC KGEC, we celebrate teamwork and attribute the success of DC to the wonderful individuals who put untiring efforts for it.
Meet our team


| Dress Alteration Prices | |
|---|---|
| Hem | $30 and up |
| Zipper | $35 and up |
| Taper | $35 and up |
| Straps | $20 and up |
| Shorten Sleeves | $25 and up |
| Taper Sleeves | $20 and up |
Trust BZ Tailor For Your Dress Alterations Every Time
Have an important event coming up that requires the perfect dress? Sometimes, a perfect fit can transform a blase dress into a one-of-a-kind piece. At BZ Tailor, dress alterations are fast, affordable and beautiful every single time, guaranteed. Don't leave your dress alterations to someone who doesn't know what they're doing. Trust BZ Tailor to transform your clothing into beautiful, timeless pieces for every occasion.
Dress Alterations Are Easier Than Ever With BZ Tailor
While many of us look for alterations specialists in convenient locations near our home or office, these specialists are not always the best choice. Sometimes, their work is rushed and messy, or it can take over a week to complete dress alterations. With BZ Tailor, the hassle of getting your clothing altered is a thing of the past. We guarantee a three-day turnaround time for our alterations. Plus, we'll pick up your clothing at your home, take measurements, and deliver it to you, so you don't have to interrupt your schedule.
Dress Alterations For Every Occasion
Whether you have a wedding coming up, prom right around the corner, or simply need a work dress nipped and tucked in the right places, we can handle it, expertly and conveniently. We pride ourselves in our exceptional work, as our in-house tailor and founder, Ricky Belopolsky, has over 40 years of experience working with all types of fabrics, garments, and clients. We're so confident in our abilities to alter any dress that if you're not completely satisfied with our work the first time around, we'll redo it at no charge. Customer satisfaction is our promise, and we'll do whatever it takes to deliver on that promise.
If you need dress alterations for work, school, a wedding or any other special event, trust BZ Tailor's expert services. After all, there's a reason we're the most trusted tailor in New York City.


Reuters is a benchmark dataset for document classification. To be more precise, it is a multi-class (e.g. there are multiple classes), multi-label (e.g. each document can belong to many classes) dataset. It has 90 classes, 7769 training documents and 3019 testing documents. It is the ModApte (R(90)) subest of the Reuters-21578 benchmark (source).
The mean number of words per document, grouped by class, is between 93 and 1263 on the training set.
The training set has a vocabulary size of 35247. Even if you restrict it to words which appear at least 5 times and at most 12672 times in the training set, there are still 12017 words.
Classes and Labels
nr of documents mean number of
class name train test words in train set
1: earn : 2877 1087 104.4
2: acq : 1650 719 150.1
3: money-fx : 538 179 219.0
4: grain : 433 149 223.6
5: crude : 389 189 247.3
6: trade : 368 117 294.3
7: interest : 347 131 198.0
8: wheat : 212 71 225.6
9: ship : 197 89 203.7
10: corn : 181 56 259.1
11: money-supply : 140 34 170.5
12: dlr : 131 44 230.4
13: sugar : 126 36 247.2
14: oilseed : 124 47 277.9
15: coffee : 111 28 264.1
16: gnp : 101 35 372.8
17: gold : 94 30 188.5
18: veg-oil : 87 37 291.0
19: soybean : 78 33 347.9
20: livestock : 75 24 222.9
21: nat-gas : 75 30 257.7
22: bop : 75 30 288.1
23: cpi : 69 28 235.4
24: cocoa : 55 18 266.4
25: reserves : 55 18 216.1
26: carcass : 50 18 259.7
27: copper : 47 18 201.6
28: jobs : 46 21 232.7
29: yen : 45 14 282.8
30: ipi : 41 12 232.1
31: iron-steel : 40 14 220.5
32: cotton : 39 20 366.3
33: barley : 37 14 272.9
34: gas : 37 17 209.4
35: rubber : 37 12 274.5
36: alum : 35 23 180.5
37: rice : 35 24 359.8
38: palm-oil : 30 10 234.5
39: meal-feed : 30 19 387.1
40: sorghum : 24 10 511.3
41: retail : 23 2 324.8
42: zinc : 21 13 189.9
43: silver : 21 8 221.0
44: pet-chem : 20 12 204.9
45: wpi : 19 10 200.9
46: tin : 18 12 322.1
47: rapeseed : 18 9 168.6
48: orange : 16 11 169.6
49: strategic-metal : 16 11 205.6
50: housing : 16 4 207.2
51: hog : 16 6 162.3
52: lead : 15 14 216.7
53: soy-oil : 14 11 568.7
54: heat : 14 5 190.4
55: fuel : 13 10 194.6
56: soy-meal : 13 13 551.0
57: lei : 12 3 134.7
58: sunseed : 11 5 425.1
59: dmk : 10 4 212.1
60: lumber : 10 6 242.1
61: tea : 9 4 365.1
62: income : 9 7 286.4
63: nickel : 8 1 193.6
64: oat : 8 6 648.4
65: l-cattle : 6 2 298.0
66: instal-debt : 5 1 134.3
67: platinum : 5 7 174.8
68: groundnut : 5 4 258.0
69: rape-oil : 5 3 167.2
70: sun-oil : 5 2 201.9
71: coconut-oil : 4 3 471.4
72: jet : 4 1 109.6
73: coconut : 4 2 264.5
74: propane : 3 3 352.0
75: potato : 3 3 161.2
76: cpu : 3 1 133.2
77: rand : 2 1 345.7
78: palmkernel : 2 1 326.3
79: copra-cake : 2 1 495.0
80: dfl : 2 1 764.7
81: naphtha : 2 4 206.7
82: palladium : 2 1 93.0
83: nzdlr : 2 2 508.8
84: groundnut-oil : 1 1 277.5
85: castor-oil : 1 1 194.0
86: sun-meal : 1 1 153.0
87: lin-oil : 1 1 262.5
88: cotton-oil : 1 2 1262.7
89: rye : 1 1 383.0
90: nkr : 1 2 122.3
By far most documents have either one or two labels, but some have up to 15:
labelcount= 1, documentcount=9160
labelcount= 2, documentcount=1173
labelcount= 3, documentcount= 255
labelcount= 4, documentcount= 91
labelcount= 5, documentcount= 52
labelcount= 6, documentcount= 27
labelcount= 7, documentcount= 9
labelcount= 8, documentcount= 7
labelcount= 9, documentcount= 5
labelcount=10, documentcount= 3
labelcount=11, documentcount= 2
labelcount=14, documentcount= 2
labelcount=12, documentcount= 1
labelcount=15, documentcount= 1
Let's look at the relationship between the classes. Which classes occur often
together? Are there classes which can be used to predict the presence of other
classes? For example, wheat
should imply grain
.
Here are the 50 strongest predictors:
castor-oil => cotton-oil (0.999742566611)
castor-oil => groundnut-oil (0.999742566611)
castor-oil => lin-oil (0.999742566611)
castor-oil => nkr (0.999742566611)
castor-oil => rye (0.999742566611)
castor-oil => sun-meal (0.999742566611)
copra-cake => palmkernel (0.999742566611)
cotton-oil => castor-oil (0.999742566611)
cotton-oil => groundnut-oil (0.999742566611)
cotton-oil => lin-oil (0.999742566611)
cotton-oil => nkr (0.999742566611)
cotton-oil => rye (0.999742566611)
cotton-oil => sun-meal (0.999742566611)
groundnut-oil => castor-oil (0.999742566611)
groundnut-oil => cotton-oil (0.999742566611)
groundnut-oil => lin-oil (0.999742566611)
groundnut-oil => nkr (0.999742566611)
groundnut-oil => rye (0.999742566611)
groundnut-oil => sun-meal (0.999742566611)
lin-oil => castor-oil (0.999742566611)
lin-oil => cotton-oil (0.999742566611)
lin-oil => groundnut-oil (0.999742566611)
lin-oil => nkr (0.999742566611)
lin-oil => rye (0.999742566611)
lin-oil => sun-meal (0.999742566611)
nkr => castor-oil (0.999742566611)
nkr => cotton-oil (0.999742566611)
nkr => groundnut-oil (0.999742566611)
nkr => lin-oil (0.999742566611)
nkr => rye (0.999742566611)
nkr => sun-meal (0.999742566611)
palmkernel => copra-cake (0.999742566611)
rye => castor-oil (0.999742566611)
rye => cotton-oil (0.999742566611)
rye => groundnut-oil (0.999742566611)
rye => lin-oil (0.999742566611)
rye => nkr (0.999742566611)
rye => sun-meal (0.999742566611)
sun-meal => castor-oil (0.999742566611)
sun-meal => cotton-oil (0.999742566611)
sun-meal => groundnut-oil (0.999742566611)
sun-meal => lin-oil (0.999742566611)
sun-meal => nkr (0.999742566611)
sun-meal => rye (0.999742566611)
castor-oil => copra-cake (0.999613849916)
castor-oil => dfl (0.999613849916)
castor-oil => naphtha (0.999613849916)
castor-oil => nzdlr (0.999613849916)
castor-oil => palladium (0.999613849916)
castor-oil => palmkernel (0.999613849916)
Multi-label Scoring
I've never been in a multi-label context, so it was not directly clear to me which scoring is used. Thanks to Chirag Nagpal who pointed me in the right direction.
Let \(Y_i \in \{0, 1\}^k\) be the set of correct labels for document \(i\) and \(Z_i \in \{0, 1\}^k\) be the set of predicted labels:
- Binary Accuracy: For each document, one has to make a decision for each possible category. Hence \(\text{acc} = \frac{1}{n}\sum_{i=1}^n \frac{|Y_i \cap Z_i|}{k}\). As most documents belong to one or two categories, the simplest classifier simply decides all the time that the document does not belong to any category. For the used dataset, this leads to an accuracy of 0.986. Hence accuracy is not suitable.
- Subset Accuracy: This is calculated by
sklearn
. The set of predicted labels must be exactly the same as the true labels. - F1 score: See user manual
- Micro/macro averaged ROC or Precision/Recall curve:
- Micro: Calculate metrics globally by counting the total true positives, false negatives and false positives.
- Macro: Calculate metrics for each label, and find their unweighted mean. This does not take label imbalance into account.
- Coverage error: See user manual
A nice overview is given by A Literature Survey on Algorithms for Multi-label Learning.
n-gram Features
See N-Gram-Based Text Categorization: Categorizing Text With Python.
I will give this a try when I find some time. If you make it before, please share the results. You could adjust my tf-idf code.
Classifier comparison (tf-idf)
The following are the accuracies as well as the training and test times. All classifiers got the same tf-idf features.
Used vocabulary size = 26147
Classifier Acc F1
--------------------------------------------------------------------------------
MLP : 82.61% 85.56% in 676.44s train / 0.91s test
LinearSVC : 81.05% 84.04% in 27.00s train / 6.45s test
Logistic Regression (C=1000) : 80.79% 84.10% in 35.48s train / 6.46s test
k nn 5 : 72.97% 76.07% in 9.89s train / 1092.29s test
k nn 3 : 72.28% 75.43% in 9.90s train / 1080.20s test
Logistic Regression (C=1) : 67.47% 67.21% in 30.22s train / 6.44s test
Random Forest (200 estimators): 65.75% 64.36% in 57.56s train / 4.15s test
Random Forest (50 estimators) : 64.79% 63.69% in 14.82s train / 1.30s test
Decision Tree : 55.75% 53.23% in 28.43s train / 0.22s test
Naive Bayes : 43.86% 47.98% in 214.75s train / 88.79s test
SVM, linear : 33.55% 29.67% in 6326.33s train / 2397.51s test
There are a couple of things to notice here:
- Speed:
- Naive Bayes and k-nn is slow
- Random forests and MLP are fast
- SVM depends extremely on the implementation (see What is the difference between LinearSVC and SVC(kernel=“linear”)?)
- Prediction Quality:
- LinearSVC, logistic regression and MLP are accurate
- Achieving high binary accuracy seems to be easier than achieving high F1 scores. It's no surprise that high subset accuracy is hard to achieve.
The MLP has a reasonable prediction quality and test time.
Multilayer Perceptron
When training a multilayer perceptron for a multi-label classification task, there are two important things to keep in mind:
- Output layer: Do not use softmax, as the normalization does not make sense in this case.
- Loss: Use
`binary_crossentropy
When you print precision, recall, F1-score and accuracy you note the following:
- Binary accuracy gets to 98% in the first epoch and over 99% in the second. It stays that high.
- Precision is at about 4% in the first epoch and over 97% in the second. It stays that high.
- Recall needs about 15 epochs of steady progress to get over 98%.
As a consequence, the F1 score steadily increases. Hence by using other metrics one can see that the classifier makes great improvements, although the accuracy is pretty high from the beginning.
Code
See GitHub.
If you use it, please cite this article or link to this blog post:
@Misc{Thom2017-reuters,
Title = {The Reuters Dataset},
Author = {Martin Thoma},
Month = jul,
Year = {2017},
Url = {https://martin-thoma.com/nlp-reuters}
}
Data loading
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
Utility file for the Reuters text categorization benchmark dataset.
See also
--------
http://www.vision.caltech.edu/Image_Datasets/Caltech101/
"""
from nltk.corpus import reuters
from nltk.corpus import stopwords
from sklearn.preprocessing import MultiLabelBinarizer
from sklearn.feature_extraction.text import TfidfVectorizer
n_classes = 90
labels = reuters.categories()
def load_data(config={}):
"""
Load the Reuters dataset.
Returns
-------
Tuple of Numpy arrays: `(x_train, y_train), (x_test, y_test)`.
"""
stop_words = stopwords.words("english")
vectorizer = TfidfVectorizer(stop_words=stop_words)
mlb = MultiLabelBinarizer()
documents = reuters.fileids()
test = [d for d in documents if d.startswith("test/")]
train = [d for d in documents if d.startswith("training/")]
docs = {}
docs["train"] = [reuters.raw(doc_id) for doc_id in train]
docs["test"] = [reuters.raw(doc_id) for doc_id in test]
xs = {"train": [], "test": []}
xs["train"] = vectorizer.fit_transform(docs["train"]).toarray()
xs["test"] = vectorizer.transform(docs["test"]).toarray()
ys = {"train": [], "test": []}
ys["train"] = mlb.fit_transform([reuters.categories(doc_id) for doc_id in train])
ys["test"] = mlb.transform([reuters.categories(doc_id) for doc_id in test])
data = {
"x_train": xs["train"],
"y_train": ys["train"],
"x_test": xs["test"],
"y_test": ys["test"],
"labels": globals()["labels"],
}
return data
if __name__ == "__main__":
config = {}
data = load_data(config)
print("len(data['x_train'])={}".format(len(data["x_train"])))
print("data['x_train'].shape={}".format(data["x_train"].shape))
MLP
#!/usr/bin/env python
"""Train and evaluate a MLP."""
import time
from keras.layers import Activation, Input, Dropout
from keras.layers import Dense
from keras.models import Model
from keras.optimizers import Adam
import reuters
from keras import backend as K
from scoring import get_tptnfpfn, get_accuracy, get_f_score
def create_model(nb_classes, input_shape):
"""Create a MLP model."""
input_ = Input(shape=input_shape)
x = input_
x = Dense(256, activation="relu")(x)
x = Dropout(0.5)(x)
x = Dense(256, activation="relu")(x)
x = Dense(nb_classes)(x)
x = Activation("sigmoid")(x)
model = Model(inputs=input_, outputs=x)
return model
def recall(y_true, y_pred):
"""
Recall metric.
Only computes a batch-wise average of recall.
Computes the recall, a metric for multi-label classification of
how many relevant items are selected.
"""
true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))
possible_positives = K.sum(K.round(K.clip(y_true, 0, 1)))
recall = true_positives / (possible_positives + K.epsilon())
return recall
def precision(y_true, y_pred):
"""
Precision metric.
Only computes a batch-wise average of precision.
Computes the precision, a metric for multi-label classification of
how many selected items are relevant.
Source
------
https://github.com/fchollet/keras/issues/5400#issuecomment-314747992
"""
true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))
predicted_positives = K.sum(K.round(K.clip(y_pred, 0, 1)))
precision = true_positives / (predicted_positives + K.epsilon())
return precision
def f1(y_true, y_pred):
"""Calculate the F1 score."""
p = precision(y_true, y_pred)
r = recall(y_true, y_pred)
return 2 * ((p * r) / (p + r))
def get_optimizer(config):
"""Return an optimizer."""
lr = config["optimizer"]["initial_lr"]
optimizer = Adam(lr=lr) # Using Adam instead of SGD to speed up training
return optimizer
def main(data_module):
"""Load data, train model and evaluate it."""
data = data_module.load_data()
model = create_model(data_module.n_classes, (data["x_train"].shape[1],))
print(model.summary())
optimizer = get_optimizer({"optimizer": {"initial_lr": 0.001}})
model.compile(
loss="binary_crossentropy",
optimizer=optimizer,
metrics=[precision, recall, f1, "accuracy"],
)
t0 = time.time()
model.fit(
data["x_train"],
data["y_train"],
batch_size=32,
epochs=20,
validation_data=(data["x_test"], data["y_test"]),
shuffle=True,
# callbacks=callbacks
)
t1 = time.time()
res = get_tptnfpfn(model, data)
t2 = time.time()
print(
(
"{clf_name:<30}: {acc:0.2f}% {f1:0.2f}% in {train_time:0.2f}s "
"train / {test_time:0.2f}s test"
).format(
clf_name="MLP",
acc=(get_accuracy(res) * 100),
f1=(get_f_score(res) * 100),
train_time=(t1 - t0),
test_time=(t2 - t1),
)
)
if __name__ == "__main__":
main(reuters)
See also
- NLTK: Other datasets
- Reuters-21578
- Publications:
- Blog posts:
- Miguel Malvarez: Classifying Reuters-21578 collection with Python: Representing the data
- Miguel Malvarez: Classifying Reuters-21578 collection with Python


{% extends "base.html" %} {% block title %}Plot Kinetics{% endblock %} {% block navbar_items %} Simulation and Tools » Plot Kinetics {% endblock %} {% block sidebar_items %} {% endblock %} {% block page_title %}Plot Kinetics{% endblock %} {% block page_body %}
Upload a chemkin file containing species thermodynamics and kinetics in order to plot both forward and reverse kinetics. You may upload an optional RMG Dictionary text file to draw the molecules and generate search links for individual reaction kinetics within the RMG database. This dictionary file must include an adjacency list for all species in the chemkin file, including inerts and collider species.


|
Divides the source operand by the destination operand and stores the result in the destination location. The destination operand (divisor) is always in an FPU register; the source operand (dividend) can be a register or a memory location. Source operands in memory can be in singleprecision or double-precision floating-point format, word or doubleword integer format.
These instructions perform the reverse operations of the FDIV, FDIVP, and FIDIV instructions.
They are provided to support more efficient coding.
The no-operand version of the instruction divides the contents of the ST(0) register by the contents of the ST(1) register. The one-operand version divides the contents of a memory location (either a floating-point or an integer value) by the contents of the ST(0) register. The twooperand version, divides the contents of the ST(i) register by the contents of the ST(0) register or vice versa.
The FDIVRP instructions perform the additional operation of popping the FPU register stack after storing the result. To pop the register stack, the processor marks the ST(0) register as empty and increments the stack pointer (TOP) by 1. The no-operand version of the floating-point divide instructions always results in the register stack being popped. In some assemblers, the mnemonic for this instruction is FDIVR rather than FDIVRP.
The FIDIVR instructions convert an integer source operand to double extended-precision floating-point format before performing the division.
If an unmasked divide-by-zero exception (#Z) is generated, no result is stored; if the exception is masked, an infinite of the appropriate sign is stored in the destination operand.
The following table shows the results obtained when dividing various classes of numbers, assuming that neither overflow nor underflow occurs.
FDIVR/FDIVRP/FIDIVR Results
| - | Destination: -inf | Destination: -F | Destination: -0 | Destination: +0 | Destination: +F | Destination: +inf | Destination: NaN |
| Source: -inf | * | +inf | +inf | -inf | -inf | * | NaN |
| Source: -F | +0 | +F | ** | ** | -F | -0 | NaN |
| Source: -I | +0 | +F | ** | ** | -F | -0 | NaN |
| Source: -0 | +0 | +0 | * | * | -0 | -0 | NaN |
| Source: +0 | -0 | -0 | * | * | +0 | +0 | NaN |
| Source: +I | -0 | -F | ** | ** | +F | +0 | NaN |
| Source: +F | -0 | -F | ** | ** | +F | +0 | NaN |
| Source: +inf | * | -inf | -inf | +inf | +inf | * | NaN |
| Source: NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN |
| F Means finite floating-point value. |
| I Means integer. |
| * Indicates floating-point invalid-arithmetic-operand (#IA) exception. |
| ** Indicates floating-point zero-divide (#Z) exception. |
[/table]When the source operand is an integer 0, it is treated as a +0.
|


gocryptfs
Encrypted overlay filesystem written in Go.
More information: home page
- Initialize an encrypted filesystem:
gocryptfs -init path/to/cipher_dir
- Mount an encrypted filesystem:
gocryptfs path/to/cipher_dir path/to/mount_point
- Mount with the explicit master key instead of password:
gocryptfs --masterkey path/to/cipher_dir path/to/mount_point
- Change the password:
gocryptfs --passwd path/to/cipher_dir
- Make an encrypted snapshot of a plain directory:
gocryptfs --reverse path/to/plain_dir path/to/cipher_dir


Here you can find a list of ontologies with the tests that the Ontology Engineering Group (OEG) is generating.
| Ontology | Category | Tests | Glossary of terms | Tests Results | Conformance with other ontologies |
|---|---|---|---|---|---|
| VICINITY Ontology model for Web of Things | IoT WoT | Tests | Glossary of terms | Testing results | |
| Vicinity core model | IoT | Tests | Glossary of terms | Testing results | |
| SAREF ontology | IoT smart appliances | Tests | Glossary of terms | Testing results |
SSN oneM2M |
| SSN ontology | IoT | Tests | Glossary of terms | Testing results |
SAREF oneM2M |
| oneM2M ontology | IoT | Tests | Glossary of terms | Testing results |
SAREF SSN |
| OpenADR | Energy demand-response | Tests | Glossary of terms | Testing results |
Here you can find conformance analysis for different domains
| Domain | Ontologies and data models included in the analysis | Shared glossary of terms | Results |
|---|---|---|---|
| IoT |
SAREF oneM2M ontology SSN ontology |
Glossary | Results |
| IoT |
SAREF oneM2M ontology SSN ontology ISO/IEC 30141 OCF |
Glossary | Results |


Small = full width, Medium = thirds, Large = sixths.
Small = half width, Medium = 1, 2, 3, 4 split, Large = 4, 3, 2, 1 split.
Small = Hide Small, Medium = Hide Medium, Large = Hide Large.