Flink and Prometheus: Cloud-native monitoring of streaming applications
March 11, 2019 - Maximilian Bode, TNG Technology Consulting (@mxpbode)This blog post describes how developers can leverage Apache Flink’s built-in metrics system together with Prometheus to observe and monitor streaming applications in an effective way. This is a follow-up post from my Flink Forward Berlin 2018 talk (slides, video). We will cover some basic Prometheus concepts and why it is a great fit for monitoring Apache Flink stream processing jobs. There is also an example to showcase how you can utilize Prometheus with Flink to gain insights into your applications and be alerted on potential degradations of your Flink jobs.
Why Prometheus? #
Prometheus is a metrics-based monitoring system that was originally created in 2012. The system is completely open-source (under the Apache License 2) with a vibrant community behind it and it has graduated from the Cloud Native Foundation last year – a sign of maturity, stability and production-readiness. As we mentioned, the system is based on metrics and it is designed to measure the overall health, behavior and performance of a service. Prometheus features a multi-dimensional data model as well as a flexible query language. It is designed for reliability and can easily be deployed in traditional or containerized environments. Some of the important Prometheus concepts are:
-
Metrics: Prometheus defines metrics as floats of information that change in time. These time series have millisecond precision.
-
Labels are the key-value pairs associated with time series that support Prometheus’ flexible and powerful data model – in contrast to hierarchical data structures that one might experience with traditional metrics systems.
-
Scrape: Prometheus is a pull-based system and fetches (“scrapes”) metrics data from specified sources that expose HTTP endpoints with a text-based format.
-
PromQL is Prometheus’ query language. It can be used for both building dashboards and setting up alert rules that will trigger when specific conditions are met.
When considering metrics and monitoring systems for your Flink jobs, there are many options. Flink offers native support for exposing data to Prometheus via the PrometheusReporter
configuration. Setting up this integration is very easy.
Prometheus is a great choice as usually Flink jobs are not running in isolation but in a greater context of microservices. For making metrics available to Prometheus from other parts of a larger system, there are two options: There exist libraries for all major languages to instrument other applications. Additionally, there is a wide variety of exporters, which are tools that expose metrics of third-party systems (like databases or Apache Kafka) as Prometheus metrics.
Prometheus and Flink in Action #
We have provided a GitHub repository that demonstrates the integration described above. To have a look, clone the repository, make sure Docker is installed and run:
./gradlew composeUp
This builds a Flink job using the build tool Gradle and starts up a local environment based on Docker Compose running the job in a Flink job cluster (reachable at http://localhost:8081) as well as a Prometheus instance (http://localhost:9090).
Job graph and custom metric for example job in Flink web interface.
The PrometheusExampleJob
has three operators: Random numbers up to 10,000 are generated, then a map counts the events and creates a histogram of the values passed through. Finally, the events are discarded without further output. The very simple code below is from the second operator. It illustrates how easy it is to add custom metrics relevant to your business logic into your Flink job.
class FlinkMetricsExposingMapFunction extends RichMapFunction<Integer, Integer> {
private transient Counter eventCounter;
@Override
public void open(Configuration parameters) {
eventCounter = getRuntimeContext().getMetricGroup().counter("events");
}
@Override
public Integer map(Integer value) {
eventCounter.inc();
return value;
}
}
Configuring Prometheus with Flink #
To start monitoring Flink with Prometheus, the following steps are necessary:
-
Make the
PrometheusReporter
jar available to the classpath of the Flink cluster (it comes with the Flink distribution):cp /opt/flink/opt/flink-metrics-prometheus-1.7.2.jar /opt/flink/lib
-
Configure the reporter in Flink’s flink-conf.yaml. All job managers and task managers will expose the metrics on the configured port.
metrics.reporters: prom metrics.reporter.prom.class: org.apache.flink.metrics.prometheus.PrometheusReporter metrics.reporter.prom.port: 9999
-
Prometheus needs to know where to scrape metrics. In a static scenario, you can simply configure Prometheus in prometheus.yml with the following:
scrape_configs: - job_name: 'flink' static_configs: - targets: ['job-cluster:9999', 'taskmanager1:9999', 'taskmanager2:9999']
In more dynamic scenarios we recommend using Prometheus’ service discovery support for different platforms such as Kubernetes, AWS EC2 and more.
Both custom metrics are now available in Prometheus:
Example metric in Prometheus web UI.
More technical metrics from the Flink cluster (like checkpoint sizes or duration, Kafka offsets or resource consumption) are also available. If you are interested, you can check out the HTTP endpoints exposing all Prometheus metrics for the job managers and the two task managers on http://localhost:9249, http://localhost:9250 and http://localhost:9251, respectively.
To test Prometheus’ alerting feature, kill one of the Flink task managers via
docker kill taskmanager1
Our Flink job can recover from this partial failure via the mechanism of Checkpointing. Nevertheless, after roughly one minute (as configured in the alert rule) the following alert will fire:
Example alert in Prometheus web UI.
In real-world situations alerts like this one can be routed through a component called Alertmanager and be grouped into notifications to systems like email, PagerDuty or Slack.
Go ahead and play around with the setup, and check out the Grafana instance reachable at http://localhost:3000 (credentials admin:flink) for visualizing Prometheus metrics. If there are any questions or problems, feel free to create an issue. Once finished, do not forget to tear down the setup via
./gradlew composeDown
Conclusion #
Using Prometheus together with Flink provides an easy way for effective monitoring and alerting of your Flink jobs. Both projects have exciting and vibrant communities behind them with new developments and additions scheduled for upcoming releases. We encourage you to try the two technologies together as it has immensely improved our insights into Flink jobs running in production.


SCORE 50 TO WIN!!
PLAYER ONE
ROLLED NUMBER:
SCORE:
ROLL
HOLD
PLAYER TWO
ROLLED NUMBER:
SCORE:
ROLL
HOLD
Play Again


module Rake::FileUtilsExt
FileUtilsExt provides a custom version of
the FileUtils methods that respond to the
verbose
and nowrite
commands.
Constants
- DEFAULT
Attributes
Public Instance Methods
Get/set the nowrite flag controlling output from the FileUtils utilities. If verbose is true, then the utility method is echoed to standard output.
Examples:
nowrite # return the current value of the # nowrite flag nowrite(v) # set the nowrite flag to _v_. nowrite(v) { code } # Execute code with the nowrite flag set # temporarily to _v_. Return to the # original value when code is done.
# File lib/rake/file_utils_ext.rb, line 78 def nowrite(value=nil) oldvalue = FileUtilsExt.nowrite_flag FileUtilsExt.nowrite_flag = value unless value.nil? if block_given? begin yield ensure FileUtilsExt.nowrite_flag = oldvalue end end oldvalue end
Check that the options do not contain options not listed in
optdecl
. An ArgumentError exception is thrown if non-declared
options are found.
# File lib/rake/file_utils_ext.rb, line 134 def rake_check_options(options, *optdecl) h = options.dup optdecl.each do |name| h.delete name end raise ArgumentError, "no such option: #{h.keys.join(' ')}" unless h.empty? end
Merge the given options with the default values.
# File lib/rake/file_utils_ext.rb, line 117 def rake_merge_option(args, defaults) if Hash === args.last defaults.update(args.last) args.pop end args.push defaults args end
Send the message to the default rake output (which is $stderr).
# File lib/rake/file_utils_ext.rb, line 127 def rake_output_message(message) $stderr.puts(message) end
Get/set the verbose flag controlling output from the FileUtils utilities. If verbose is true, then the utility method is echoed to standard output.
Examples:
verbose # return the current value of the # verbose flag verbose(v) # set the verbose flag to _v_. verbose(v) { code } # Execute code with the verbose flag set # temporarily to _v_. Return to the # original value when code is done.
# File lib/rake/file_utils_ext.rb, line 54 def verbose(value=nil) oldvalue = FileUtilsExt.verbose_flag FileUtilsExt.verbose_flag = value unless value.nil? if block_given? begin yield ensure FileUtilsExt.verbose_flag = oldvalue end end FileUtilsExt.verbose_flag end
Use this function to prevent potentially destructive ruby code from running when the :nowrite flag is set.
Example:
when_writing("Building Project") do project.build end
The following code will build the project under normal conditions. If the nowrite(true) flag is set, then the example will print:
DRYRUN: Building Project
instead of actually building the project.
# File lib/rake/file_utils_ext.rb, line 108 def when_writing(msg=nil) if FileUtilsExt.nowrite_flag $stderr.puts "DRYRUN: #{msg}" if msg else yield end end


nf-core
The nf-core framework tools, to create, check and develop best-practice guidelines for Nextflow.
More information: home page
- List existing pipelines on nf-core:
nf-core list
- Create a new pipeline skeleton:
nf-core create
- Lint the pipeline code:
nf-core lint path/to/directory
- Bump software versions in pipeline recipe:
nf-core bump-version path/to/directory new_version
- Launch an nf-core pipeline:
nf-core launch pipeline_name
- Download an nf-core pipeline for offline use:
nf-core download pipeline_name


HAMLET A film review by Frank Maloney Copyright 1991 Frank Maloney
Two Shakespeare plays translated to film in one year is my idea of a good year. I liked HENRY V despite the staginess and some of other cavils. Likewise, I liked HAMLET (with reservations).
Now I know the movie (and opera) world divides pretty neatly into those who like Franco Zeffirelli's work and those who abominate him and it. Likewise, the world of Shakespeare divides into purists who sanctify the full and complete received texts and those who don't mind having a bit a fun here and there and aren't above playing fast and loose with the Master.

Zeffirelli's HAMLET is no more Shakespeare's HAMLET than his ROMEO AND JULIET or his TRAVIATA was anybody's but his own. What I like about his work is that he understands the difference between a play and a movie. His HAMLET is absolutely, positively a movie and not a filmed play. And movies always muck about with the source material, very much as Shakespeare mucked about with his source material. Even in the titles, it doesn't say "Shakespeare's Hamlet"; it says, quite honestly, "Based on the play by William Shakespeare."
Zeffirelli has excised scenes, moved scenes around, relocated scenes. Almost every fan of Shakespeare's HAMLET will find a favorite scene missing or altered. There is even added material such as the executions of Rosencrantz and Guildenstern and the death of Ophelia. These edits seem to me to be fully justified in the name of making a movie out of a play.
Of course, most of us will agree that Zeffirelli has a vast and dramatic visual sense, whether or not we can agree that he always uses that visual sense with the best of taste. He uses locales, sets, and costumes with exuberance and care. And it makes this movie, like his others, visual feasts. He also shoots many of his scenes en plein air, others indoors in something like available light. His camera is busy, perhaps busier than is strictly necessary or desirable, but that may be preferable to long static shots of actors declaiming (no names, please).
Another thing to like about Zeffirelli's HAMLET is the diction of the actors. They speak their parts more naturally, less hurriedly, and swallow less of the words than is the general rule with Shakespearean productions.
Without a doubt, I understand HAMLET better now that I've seen Zeffirelli's version. I've studied it in school and seen live and filmed versions, but I missed, not being a great Shakespearean scholar even in my grad school days, some basic structural details that came through loud and clear this time for the first time. Par exemple, just as the play-within-the-play mirrors the major action, so too does Laertes mirror Hamlet and his situation when he lets Claudius manipulate him into killing Hamlet but with the telling difference that Laertes never questions Claudius's honesty or motives and proceeds without scruples to do his treachery. Probably everyone else in the civilized world knew all about this; to me it came as a revelation while watching the final scene in the movie.
And there are some fine actors in this movie. I don't have the cast list in front of me so rather than bollix the actors' names as is my wont, let me just say that the Ghost, Polonius, Laertes, and Guildenstern & Rosencrantz (one of whom was almost certainly a Maloney, by the way) were all first rate; the Polonius was especially fine, in my humble opinion, balancing comedy and an underlying seriousness with great deftness and precision. And the Ghost made the most of his small role with some fine physical shtick, especially one gesture where he covers half his face by fanning it with a kind of broken wrist movement as he tells Hamlet he is forbidden to speak of the afterlife: a gem of acting.
I was almost as impressed by the acting of the "name" stars, Helena Bonham-Carter, Glenn Close, and Mel Gibson, in descending order of impressiveness.  Close looked marvelous and her scene with her son that ends with a hint of incest was strong and moving; it had me in tears. Gibson, a Shakespearean dark horse if ever there were one, was surprisingly good in most of his scenes with other actors; the art of the soliloquy, however, has eluded him so far. He moves well, brings a brooding energy to his part, and did not embarrass. There were times, though, when I feared his eyes would pop right out of his head, which is distracting at the least. And I have to say that he was too old for the part, at least as I envision Hamlet the young scholar-prince.
Part of Gibson's trouble with the soliloquies perhaps derives from Zeffirelli's direction. The movie has tendency to pause and almost clear its throat before launching into a famous speech; at times the quotation marks were nearly visible. I think any director has to work very hard to overcome the Barlett's-Familiar-Quotations aspect of HAMLET. Too often, I felt Zeffirelli instead of downplaying this distracting aspect was playing it up, as if he had just realized "Oh, that's were that comes from."
Overall, despite its shortcomings, I don't think you can regret going to Zeffirelli's HAMLET.
-- Frank Richard Aloysius Jude Maloney
.
The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


| DOSE: |
1.5 tablets |
oral |
MDMA |
(pill / tablet) |
I live in MN and every now and then my family and i go on vacation to the beach in NC. Well this year I brought my cousin along with me. One night we happen to fall into a conversation about X. I had done it a few times but never too much. I never thought that she had done it before. I was wrong. And what she told me was the most beautiful story .
About 2 years ago, her little sister had died at the age of 9. She stuggled with it for so long. One day a friend of hers stopped over and gave her a pill that she didn't want. My cousin took half and went for a drive not knowing what to expect. She told me that while she was driving around she started to think about her sister and for the first time in 2 years it made her smile. Instead of thinking about the fact that she died, she thought about all the good times she had with her. That night she came to terms with her sisters death! I was so shocked and it made me cry. I had never heard of anyone having such a dramatic experience before like that.
Our last night there we were looking for some people to hang out with when this kid comes up to us and asks us if we want to buy a couple rolls. We decide to go for it and buy 3 pills for the 2 of us. We sat on the beach at night and watched baby sea turtles hatch and make their way to the ocean. It was the most spiritual feeling I have ever had. The way that we both feel about the presence of new life beginning. And the things that you'll encounter when you're faced with death. And how one little pill can change your thoughts about it in a good way. We looked at each other without saying a word, smiled, and we both knew we felt the same. It gave me goosebumps. Thank you very much.
Experience Reports are the writings and opinions of the individual authors who submit them.
Some of the activities described are dangerous and/or illegal and none are recommended by Erowid.


|
Birth
|
1920-03-13 |
|
Plains Pennsylvania, possibly a home birth
|
|
|
|
|
|
Death
|
2009-08-27 |
|
Wooster, Massacusetts (University of Massachusetts Hospital).
|
|
|
|
|
|
Burial
|
2009-09-02 |
|
Immaculate Conception (St Mary's Cemetary), Lawrence, Massachusetts
|
|
|
|
|
|
Marriage
|
1948-06-28 |
|
St. Peter and Pauls Church, Plains, Pennsylvania
|
|
|
|
|
|
Education
|
1941 |
|
Angela Pauline Hajduk (later Naber, and her legal name at the being Elenora Hajduk: more information in the name section) graduated in 1941 from Misericordia College with a BA in Latin.
|
|
|
|
Event Note
Notes about Angela's college years:
- She majored in Latin, earned a BA, and graduated in 1941.
- She was a member of
- Mendel Science Club (and president of the club during her senior year: 1940-1941)
- Academic Club
- Liturgial Choir (which performed religious songs)
- She was also involved in a lot of other activites and events, including in attached newspaper clippings.
|


Creates sophisticated models of training data and validates the models with an independent test set, cross validation, or Out Of Bag (OOB) predictions on the training data. Create graphs and tables of the model validation results. Applies these models to GIS .img files of predictors to create detailed prediction surfaces. Handles large predictor files for map making, by reading in the .img files in chunks, and output to the .txt file the prediction for each data chunk, before reading the next chunk of data.
| Version: | <IP_ADDRESS> |
| Depends: | R (≥ 2.13.0), randomForest, raster, rgdal |
| Imports: | graphics, grDevices, stats, utils, mgcv, corrplot, fields, HandTill2001, PresenceAbsence |
| Suggests: | party, quantregForest |
| Published: | 2018-09-11 |
| Author: | Elizabeth Freeman, Tracey Frescino |
| Maintainer: | Elizabeth Freeman <eafreeman at fs.fed.us> |
| License: | Unlimited |
| Copyright: | see file COPYRIGHTS |
| NeedsCompilation: | no |
| Citation: | ModelMap citation info |
| Materials: | ChangeLog |
| In views: | Spatial |
| CRAN checks: | ModelMap results |
| Reference manual: | ModelMap.pdf |
| Vignettes: |
Exploratory Data Analysis with the ModelMap package ModelMap: an R package for Model Creation and Map Production Pick your flavor of Random Forest |
| Package source: | ModelMap_<IP_ADDRESS>.tar.gz |
| Windows binaries: | r-devel: ModelMap_<IP_ADDRESS>.zip, r-release: ModelMap_<IP_ADDRESS>.zip, r-oldrel: ModelMap_<IP_ADDRESS>.zip |
| OS X binaries: | r-release: ModelMap_<IP_ADDRESS>.tgz, r-oldrel: ModelMap_<IP_ADDRESS>.tgz |
| Old sources: | ModelMap archive |
Please use the canonical form https://CRAN.R-project.org/package=ModelMap to link to this page.


Precision medicine refers to the use of prevention and treatment strategies that are tailored to the unique features of each individual and their disease. In the context of cancer this might involve the identification of specific mutations shown to predict response to a targeted therapy. The biomedical literature describing these associations is large and growing rapidly. Realizing precision medicine will require this information to be centralized, debated and interpreted for application in the clinic. Currently these interpretations exist largely in private or encumbered databases resulting in extensive repetition of effort. CIViC is an open access, open source, community-driven web resource for Clinical Interpretation of Variants in Cancer. Our goal is to enable precision medicine by providing an educational forum for dissemination of knowledge and active discussion of the clinical significance of cancer genome alterations.
Development Disclaimer: CIViC is under heavy development, therefore you will more likely than not experience bugs, unexpected behavior, missing features, and general instability. We will also be periodically dropping and re-importing the database, so any comments or updates you make while CIViC is under development will likely be lost.
Medical Disclaimer: This resource is intended for purely research purposes. It should not be used for emergencies or medical or professional advice.
CIViC is a community-edited forum for discussion and interpretation of peer-reviewed publications pertaining to the clinical relevance of variants (or biomarker alterations) in cancer. These interpretations may include associations between molecular alterations (or lack of alteration) and one or more drugs, diagnoses, prognoses or other treatment decisions. These interpretations of clinical significance (or lack of clinical significance) are purely for research purposes. A finding of no interpretation does not necessarily indicate lack of relevance for any specific variant or biomarker alteration. Interpretations are not presented in ranked order of potential or predicted importance.These interpretations make no promise or guarantee of any clinical benefit (or lack of clinical benefit).
The civicdb.org (civic.genome.wustl.edu) website does not provide any medical or healthcare products, services or advice, and is not for medical emergencies or urgent situations.  Information contained on this website is not a substitute for a doctor's medical judgment or advice. We recommend that you discuss your specific, individual health concerns with your doctor or health care professional.


Deadpool 2
Deadpool 2 is a movie starring Ryan Reynolds, Josh Brolin, and Morena Baccarin. Foul-mouthed mutant mercenary Wade Wilson (a.k.a. Deadpool), brings together a team of fellow mutant rogues to protect a young boy with supernatural...


LIFX launches Black Friday deals with crazy bundles and more
Along with the bundles, LIFX is selling individual lights for up to 33 percent off
by Jonathan LamontLIFX announced several deals that will run through Black Friday and Cyber Monday. If you’re looking to pick up some smart lights, now may be the time to do it.
Below are the deals, which are available on the company’s website:
- Bedroom Bundle: 2 x Day & Dusk, 1 x A19 – $79 (regular $119.97)
- Gamer Bundle: 2 x Mini Colour, 1 x Single Tile Kit, 1 x Z LED Kit – $149 (regular $259.96)
- Smart Starter Bundle: 4 x A19, 2 x Mini Colour – $230 (regular $329.94)
- The Massive Bundle: 2 x BR30+, 4 x BR30, 4 X GU10, 4 x Mini Colour, 2 x Mini Day & Dusk, 1 x Mini White – $599 (regular $899.93)
Of course, individual bulbs are available on sale as well, with LIFX boasting up to 33 percent off select items. Deals run until December 2nd.
You can learn more about the deals on the LIFX website.
Update 11/25/2019 5:50: Corrected some errors in the article about discount amounts.


Vanilla JS:
npm i --save @shaai/scroll-ink
React:
npm i --save @shaai/scroll-ink @shaai/react
✐ Read our blog


//CC0 / Creative Commons Zero v1.0 Universal
x
Hello World
x
To the extent possible under law,
Venni Härkönen
has waived all copyright and related or neighboring rights to
venninweb .
This work is published from:
Suomi.