For git credential approve
git (2.28.0 macOS) does not even call the credential helper if no username is supplied:
The credential.helper
key has a multi-set value, so if you add a new value, the old values are still kept. From git 2.9 specifying an empty string removes the previously defined helpers.
We do this with an eye of supporting usernames and multiple users.
cache
This helper is not included in the default git installer on Windows . :(
This is how we can set up cache
for a particular repository:
❯ mkdir foo
❯ cd foo
❯ git init .
11:43:28.618841 git.c:444 trace: built-in: git init .
Initialized empty Git repository in /private/tmp/foo/.git/
❯ git config --add credential.helper ""
11:43:50.682962 git.c:444 trace: built-in: git config --add credential.helper ''
❯ git config --add credential.helper cache
11:43:54.577707 git.c:444 trace: built-in: git config --add credential.helper cache
❯ cat .git/config
[core]
repositoryformatversion = 0
filemode = true
bare = false
logallrefupdates = true
ignorecase = true
precomposeunicode = true
[credential]
helper =
helper = cache
Now let’s add credentials:
❯ (echo url=https://github.com; echo username=token; echo password=secret; echo ) | git credential approve
11:45:16.813913 git.c:444 trace: built-in: git credential approve
11:45:16.814431 run-command.c:663 trace: run_command: 'git credential-cache store'
11:45:16.823008 git.c:704 trace: exec: git-credential-cache store
11:45:16.823637 run-command.c:663 trace: run_command: git-credential-cache store
11:45:16.842902 run-command.c:663 trace: run_command: git-credential-cache--daemon /Users/gaborcsardi/.cache/git/credential/socket
❯ (echo url=https://github.com; echo username=token2; echo password=secret2; echo ) | git credential approve
11:45:28.927712 git.c:444 trace: built-in: git credential approve
11:45:28.928108 run-command.c:663 trace: run_command: 'git credential-cache store'
11:45:28.937087 git.c:704 trace: exec: git-credential-cache store
11:45:28.937695 run-command.c:663 trace: run_command: git-credential-cache store
Query with username works correctly:
❯ (echo<EMAIL_ADDRESS>echo ) | git credential fill
11:46:40.689122 git.c:444 trace: built-in: git credential fill
11:46:40.689638 run-command.c:663 trace: run_command: 'git credential-cache get'
11:46:40.696784 git.c:704 trace: exec: git-credential-cache get
11:46:40.697333 run-command.c:663 trace: run_command: git-credential-cache get
protocol=https
host=github.com
username=token
password=secret
❯ (echo<EMAIL_ADDRESS>echo ) | git credential fill
11:46:43.767002 git.c:444 trace: built-in: git credential fill
11:46:43.767676 run-command.c:663 trace: run_command: 'git credential-cache get'
11:46:43.778637 git.c:704 trace: exec: git-credential-cache get
11:46:43.779201 run-command.c:663 trace: run_command: git-credential-cache get
protocol=https
host=github.com
username=token2
password=secret2
Query without username works, and returns some credential:
❯ (echo url=https://github.com; echo ) | git credential fill
11:45:58.200272 git.c:444 trace: built-in: git credential fill
11:45:58.200667 run-command.c:663 trace: run_command: 'git credential-cache get'
11:45:58.208372 git.c:704 trace: exec: git-credential-cache get
11:45:58.208919 run-command.c:663 trace: run_command: git-credential-cache get
protocol=https
host=github.com
username=token
password=secret
❯ (echo<EMAIL_ADDRESS>echo ) | git credential reject
11:47:03.921697 git.c:444 trace: built-in: git credential reject
11:47:03.922530 run-command.c:663 trace: run_command: 'git credential-cache erase'
11:47:03.935858 git.c:704 trace: exec: git-credential-cache erase
11:47:03.936400 run-command.c:663 trace: run_command: git-credential-cache erase
❯ (echo url=https://github.com; echo ) | git credential fill
11:47:10.018877 git.c:444 trace: built-in: git credential fill
11:47:10.019386 run-command.c:663 trace: run_command: 'git credential-cache get'
11:47:10.027990 git.c:704 trace: exec: git-credential-cache get
11:47:10.028572 run-command.c:663 trace: run_command: git-credential-cache get
protocol=https
host=github.com
username=token2
password=secret2
store
Configure for a repo:
❯ mkdir foo
❯ cd foo
❯ git init .
11:53:48.042569 git.c:444 trace: built-in: git init .
Initialized empty Git repository in /private/tmp/foo/.git/
❯ git config --add credential.helper ""
11:53:52.949914 git.c:444 trace: built-in: git config --add credential.helper ''
❯ git config --add credential.helper store
11:53:56.682348 git.c:444 trace: built-in: git config --add credential.helper store
❯ cat .git/config
[core]
repositoryformatversion = 0
filemode = true
bare = false
logallrefupdates = true
ignorecase = true
precomposeunicode = true
[credential]
helper =
helper = store
Add credentials:
❯ (echo url=https://github.com; echo username=token; echo password=secret; echo ) | git credential approve
11:54:44.184929 git.c:444 trace: built-in: git credential approve
11:54:44.185729 run-command.c:663 trace: run_command: 'git credential-store store'
11:54:44.197920 git.c:704 trace: exec: git-credential-store store
11:54:44.198471 run-command.c:663 trace: run_command: git-credential-store store
/tmp/foo master
❯ (echo url=https://github.com; echo username=token2; echo password=secret2; echo ) | git credential approve
11:54:48.452942 git.c:444 trace: built-in: git credential approve
11:54:48.453399 run-command.c:663 trace: run_command: 'git credential-store store'
11:54:48.463535 git.c:704 trace: exec: git-credential-store store
11:54:48.464004 run-command.c:663 trace: run_command: git-credential-store store
Query with username:
❯ (echo<EMAIL_ADDRESS>echo ) | git credential fill
11:55:21.191654 git.c:444 trace: built-in: git credential fill
11:55:21.192357 run-command.c:663 trace: run_command: 'git credential-store get'
11:55:21.204279 git.c:704 trace: exec: git-credential-store get
11:55:21.205063 run-command.c:663 trace: run_command: git-credential-store get
protocol=https
host=github.com
username=token
password=secret
❯ (echo<EMAIL_ADDRESS>echo ) | git credential fill
11:55:24.194096 git.c:444 trace: built-in: git credential fill
11:55:24.194654 run-command.c:663 trace: run_command: 'git credential-store get'
11:55:24.207028 git.c:704 trace: exec: git-credential-store get
11:55:24.207643 run-command.c:663 trace: run_command: git-credential-store get
protocol=https
host=github.com
username=token2
password=secret2
Query without username returns some credentials, apparently not the ones that were set first:
❯ (echo url=https://github.com; echo ) | git credential fill
11:56:12.394594 git.c:444 trace: built-in: git credential fill
11:56:12.394949 run-command.c:663 trace: run_command: 'git credential-store get'
11:56:12.403303 git.c:704 trace: exec: git-credential-store get
11:56:12.403863 run-command.c:663 trace: run_command: git-credential-store get
protocol=https
host=github.com
username=token2
password=secret2
❯ (echo<EMAIL_ADDRESS>echo ) | git credential reject
11:56:24.065910 git.c:444 trace: built-in: git credential reject
11:56:24.066314 run-command.c:663 trace: run_command: 'git credential-store erase'
11:56:24.074851 git.c:704 trace: exec: git-credential-store erase
11:56:24.076875 run-command.c:663 trace: run_command: git-credential-store erase
❯ (echo url=https://github.com; echo ) | git credential fill
11:56:26.438444 git.c:444 trace: built-in: git credential fill
11:56:26.438839 run-command.c:663 trace: run_command: 'git credential-store get'
11:56:26.446181 git.c:704 trace: exec: git-credential-store get
11:56:26.446721 run-command.c:663 trace: run_command: git-credential-store get
protocol=https
host=github.com
username=token
password=secret
osxkeychain
Some docs: https://docs.github.com/en/github/using-git/updating-credentials-from-the-osx-keychain
This is the default helper on macOS currently (git 2.28.0).
This is how it stores a credential:
It installs as a git subcommand, so it is possible to call its internal api directly:
❯ git credential-osxkeychain
11:50:56.325499 git.c:704 trace: exec: git-credential-osxkeychain
11:50:56.325783 run-command.c:663 trace: run_command: git-credential-osxkeychain
usage: git credential-osxkeychain <get|store|erase>
As always, needs username
when setting the credential.
No need to supply username
to get some token that matches the host. This is in a clean keychain. First we set two credentials:
❯ (echo url=https://github.com; echo username=token; echo password=secret; echo ) | git credential approve
10:48:47.187164 git.c:444 trace: built-in: git credential approve
10:48:47.187691 run-command.c:663 trace: run_command: 'git credential-osxkeychain store'
10:48:47.197964 git.c:704 trace: exec: git-credential-osxkeychain store
10:48:47.198518 run-command.c:663 trace: run_command: git-credential-osxkeychain store
❯ (echo url=https://github.com; echo username=token2; echo password=secret2; echo ) | git credential approve
10:48:55.299933 git.c:444 trace: built-in: git credential approve
10:48:55.300282 run-command.c:663 trace: run_command: 'git credential-osxkeychain store'
10:48:55.308568 git.c:704 trace: exec: git-credential-osxkeychain store
10:48:55.309276 run-command.c:663 trace: run_command: git-credential-osxkeychain store
If we don’t supply a username, then we’ll just get one of them:
❯ (echo url=https://github.com; echo ) | git credential fill
10:49:17.371636 git.c:444 trace: built-in: git credential fill
10:49:17.372021 run-command.c:663 trace: run_command: 'git credential-osxkeychain get'
10:49:17.378688 git.c:704 trace: exec: git-credential-osxkeychain get
10:49:17.379164 run-command.c:663 trace: run_command: git-credential-osxkeychain get
protocol=https
host=github.com
username=token
password=secret
If we supply the username, then we’ll get the correct one:
❯ (echo<EMAIL_ADDRESS>echo ) | git credential fill
10:49:28.613779 git.c:444 trace: built-in: git credential fill
10:49:28.614108 run-command.c:663 trace: run_command: 'git credential-osxkeychain get'
10:49:28.621440 git.c:704 trace: exec: git-credential-osxkeychain get
10:49:28.621979 run-command.c:663 trace: run_command: git-credential-osxkeychain get
protocol=https
host=github.com
username=token2
password=secret2
To check that without a username we get an arbitrary one, let’s remove token
:
❯ (echo<EMAIL_ADDRESS>echo ) | git credential reject
10:49:58.584332 git.c:444 trace: built-in: git credential reject
10:49:58.586880 run-command.c:663 trace: run_command: 'git credential-osxkeychain erase'
10:49:58.598463 git.c:704 trace: exec: git-credential-osxkeychain erase
10:49:58.599214 run-command.c:663 trace: run_command: git-credential-osxkeychain erase
```sh
❯ (echo url=https://github.com; echo ) | git credential fill
10:50:07.468385 git.c:444 trace: built-in: git credential fill
10:50:07.468728 run-command.c:663 trace: run_command: 'git credential-osxkeychain get'
10:50:07.478398 git.c:704 trace: exec: git-credential-osxkeychain get
10:50:07.478832 run-command.c:663 trace: run_command: git-credential-osxkeychain get
protocol=https
host=github.com
username=token2
password=secret2
Now let’s re-add token to make sure that osxkeychain
does not prefer token
:
❯ (echo url=https://github.com; echo username=token; echo password=secret; echo ) | git credential approve
10:58:52.302066 git.c:444 trace: built-in: git credential approve
10:58:52.311063 run-command.c:663 trace: run_command: 'git credential-osxkeychain store'
10:58:52.321633 git.c:704 trace: exec: git-credential-osxkeychain store
10:58:52.322108 run-command.c:663 trace: run_command: git-credential-osxkeychain store
❯ (echo url=https://github.com; echo ) | git credential fill
10:58:57.316418 git.c:444 trace: built-in: git credential fill
10:58:57.317630 run-command.c:663 trace: run_command: 'git credential-osxkeychain get'
10:58:57.330142 git.c:704 trace: exec: git-credential-osxkeychain get
10:58:57.330697 run-command.c:663 trace: run_command: git-credential-osxkeychain get
protocol=https
host=github.com
username=token2
password=secret2
So it seems that osxkeychain
will just find an arbitrary one, or the one that was added first.
manager-core
(on macOS), before version 2.0.246-betaNot installed by default (git 2.28.0).
Install from brew, according to the instructions: https://github.com/microsoft/Git-Credential-Manager-Core#macos-homebrew
If updates your global git config, adding these lines:
[credential]
helper = ""
[credential "https://dev.azure.com"]
useHttpPath = true
[credential]
helper = manager-core
The helper = ""
line deletes previous handlers. The system helper is kept as osxkeychain
.
It installs as a git subcommand, and you can call its internal API directly:
❯ git credential-manager-core
11:51:56.434300 git.c:704 trace: exec: git-credential-manager-core
11:51:56.434496 run-command.c:663 trace: run_command: git-credential-manager-core
Missing command.
usage: git-credential-manager-core <command>
Available commands:
erase
get
store
configure [--system]
unconfigure [--system]
--version, version
--help, -h, -?
It is not compatible with the osxkeychain
helper, because it uses different keys in the keychain.
It supports different providers. Providers are auto-detected by default. GitHub has its own provider, detected via the github.com
URL. The provider can be configured user a git config key or an environment variable: https://github.com/microsoft/Git-Credential-Manager-Core/blob/master/docs/configuration.md#credentialprovider
It does not currently supports namepaces (like manager
).
This is how it stores a credential:
Name: git:https://github.com
Kind: application password
Account: token
Where: git:https://github.com
No need to supply username
to get some credential:
❯ (echo url=https://github.com; echo ) | git credential fill
11:24:47.750966 git.c:444 trace: built-in: git credential fill
11:24:47.753268 run-command.c:663 trace: run_command: 'git credential-manager-core get'
11:24:47.762249 git.c:704 trace: exec: git-credential-manager-core get
11:24:47.762917 run-command.c:663 trace: run_command: git-credential-manager-core get
protocol=https
host=github.com
username=token
password=secret
If there are multiple credentials, a random one (or the one set first?) is returned:
❯ (echo url=https://github.com; echo username=token2; echo password=secret2; echo ) | git credential approve
11:25:41.553761 git.c:444 trace: built-in: git credential approve
11:25:41.554242 run-command.c:663 trace: run_command: 'git credential-manager-core store'
11:25:41.565748 git.c:704 trace: exec: git-credential-manager-core store
11:25:41.566218 run-command.c:663 trace: run_command: git-credential-manager-core store
In fact the username is completely ignored when getting credentials, at least for the GitHub provider:
❯ (echo<EMAIL_ADDRESS>echo ) | git credential fill
11:29:49.274574 git.c:444 trace: built-in: git credential fill
11:29:49.275020 run-command.c:663 trace: run_command: 'git credential-manager-core get'
11:29:49.283563 git.c:704 trace: exec: git-credential-manager-core get
11:29:49.284236 run-command.c:663 trace: run_command: git-credential-manager-core get
protocol=https
host=github.com
username=token
password=secret
This is because the username is not stored as part of the URL by the GitHub provider. Related issue: https://github.com/microsoft/Git-Credential-Manager-Core/issues/160 manager
has a similar problem, it is linked from this GitHub issue.
If you set the provider to Generic, then usernames work as expected. In this case, this is stored in the keychain:
Name<EMAIL_ADDRESS>Kind: application password
Account: token
Where<EMAIL_ADDRESS>These credentials are not compatible with the ones set by the GitHub provider.
❯ export GCM_PROVIDER=generic
❯ (echo url=https://github.com; echo username=token; echo password=secret; echo ) | git credential approve
11:34:15.998644 git.c:444 trace: built-in: git credential approve
11:34:15.998992 run-command.c:663 trace: run_command: 'git credential-manager-core store'
11:34:16.008178 git.c:704 trace: exec: git-credential-manager-core store
11:34:16.008834 run-command.c:663 trace: run_command: git-credential-manager-core store
❯ (echo url=https://github.com; echo username=token2; echo password=secret2; echo ) | git credential approve
11:35:52.629963 git.c:444 trace: built-in: git credential approve
11:35:52.637966 run-command.c:663 trace: run_command: 'git credential-manager-core store'
11:35:52.648058 git.c:704 trace: exec: git-credential-manager-core store
11:35:52.648514 run-command.c:663 trace: run_command: git-credential-manager-core store
❯ (echo<EMAIL_ADDRESS>echo ) | git credential fill
11:35:58.336428 git.c:444 trace: built-in: git credential fill
11:35:58.336881 run-command.c:663 trace: run_command: 'git credential-manager-core get'
11:35:58.345187 git.c:704 trace: exec: git-credential-manager-core get
11:35:58.345729 run-command.c:663 trace: run_command: git-credential-manager-core get
protocol=https
host=github.com
username=token
password=secret
❯ (echo<EMAIL_ADDRESS>echo ) | git credential fill
11:36:02.550339 git.c:444 trace: built-in: git credential fill
11:36:02.550695 run-command.c:663 trace: run_command: 'git credential-manager-core get'
11:36:02.557777 git.c:704 trace: exec: git-credential-manager-core get
11:36:02.558359 run-command.c:663 trace: run_command: git-credential-manager-core get
protocol=https
host=github.com
username=token2
password=secret2
manager-core
(macOS), 2.0.246-beta or laterusername
as part of the service any more, for the generic provider.username
as account
and it uses it when searching for credentials, both for the generic
and the github
provider.manager-core
(on Windows, before version 2.0.246-beta)Not installed by default (git 2.28.0), but this is supposed to be the future canonical implementation. See above for the macOS version.
It works similarly to manager
, I haven’t seen any differences in behavior (version was <IP_ADDRESS>577). Instead of authorities, we need to set up providers: https://github.com/microsoft/Git-Credential-Manager-Core/blob/master/docs/migration.md#gcm_authority. The Generic
provider works the same way as the Basic
authority in manager
.
Unfortunately setting GCM_AUTHORITY
will make manager-core
break, so it is not possible to use both manager
and manager-core
if you need to set this environment variable.
manager-core
(on Windows, from 2.0.246-beta)manager
is still installed, but the default system config sets manager-core
.username
field of the credential, still. so it is not lost.Use the osxkeychain
helper.
FIrst remove all your current credentials for the host you are targeting. E.g. for GitHub, search for “Internet Passwords” for github.com, or use gitcreds::gitcreds_list()
and the oskeyring package to remove them. You can also use the oskeyring package to back up the tokens and passwords.
Then add the credential that you want to use for “generic access”. This is the credential that will be used for URLs without user names. The user name for this credential does not matter, but you can choose something descriptive, e.g. “PersonalAccessToken”, “token”, or “generic”.
Configure git to use this username by default. E.g. if you chose “generic”, then run
git config --global crendetial.username generic
Add all the other credentials, with appropriate user names. These are the user names that you need to put in the URLs for the repositories or operations you want to use them for. (GitHub does not actually use the user names if the password is a PAT.)
We suggest that you update to the latest git version, but at least 2.29.0, and use the manager-core
helper which is now default. If you installed manager-core
separately from git, we suggest that you remove it, because it might cause confusion as to which helper is actually used.
Remove all current credentials first, for the host you are targeting. You can do this in ‘Credential Manager’ or gitcreds::gitcreds_list()
to find them and the oskeyring package to remove them. You can also use the oskeyring packaeg to back up the tokens and passwords.
Then add the credential that you want to use for “generic access”. This is the credential that will be used for URLs without user names. The user name for this credential does not matter, but you can choose something descriptive, e.g. “PersonalAccessToken”, “token”, or “generic”.
Configure git to use this username by default. E.g. if you chose “generic”, then run
git config –global crendetial.username generic
Add all the other credentials, with appropriate user names. These are the user names that you need to put in the URLs for the repositories or operations you want to use them for. (GitHub does not actually use the user names if the password is a PAT.)
If you only need to manage a single github.com credential, together with possibly multiple credentials to other hosts (including GitHub Enterprise hosts), then you can use the default manager
helper, and get away with the default auto-detected GCM authority setting.
In this case, you can add you github.com credential with an arbitrary user name, and for each other host you can add configure a default user name, and/or include user names in the URLs to these hosts. This is how to set a default user name for a host:
git config --global credential.https://example.com.username myusername
If you need to manage multiple github.com credentials, then you can still use the manager
helper, but you need to change the GCM authority by setting an option or an environment variable, see https://github.com/microsoft/Git-Credential-Manager-for-Windows/blob/master/Docs/Configuration.md#authority. Once https://github.com/microsoft/Git-Credential-Manager-for-Windows/pull/891 is merged, you won’t need to do this. (At least in recent git versions, that contain a GCM build with the fix.)
This is how to change the config for this:
git config --global credential.authority Basic
You can also change it only for github.com:
git config --global credential.github.com.authority Basic
Then you can configure a default user name, this will be used for URLs without a user name:
git config --global credential.username generic
Now you can add you credentials, the default one with the “generic” user name, and all the others with their specific user and host names.


Target Just Sent Out Its Black Friday Ad — We Have All the Details!
by Rebecca BrownIf you're already thinking about your holiday shopping and all the potential Black Friday deals to be had, you'll be happy to know that Target just introduced a new promotion called "HoliDeals" this morning, featuring days and days of savings on electronics, toys, and more. Here's what you can expect from Target's HoliDeals:
- Black Friday Two-Day Preview Sale (Friday, Nov. 8 – Saturday, Nov. 9): Guests will be able to shop some of the best deals from Target's Black Friday ad. This year, Target has quadrupled the number of deals and is extending it to two days!
- Expanded Early Access to Black Friday Deals (Wednesday, Nov. 27): Later in the month, Target will offer early access to some of its best Black Friday deals to Target RedCard holders and Target Circle members.
- Black Friday Savings and Hours (Thursday, Nov. 28 – Saturday, Nov. 30): The sale we've all been waiting for! Shoppers can start taking advantage of Black Friday savings beginning Thanksgiving morning on Target.com and in Target stores at 5 p.m. on Thursday, Nov. 28 (remaining open until 1 a.m. on Friday, Nov. 29). Stores will reopen on Friday morning at 7 a.m.
- Cyber Week Savings (Sunday, Dec. 1 – Sunday, Dec. 8)
- Green Monday (Monday, Dec. 9)


Want this for your office? Scroll down to find out more
This office location and many other locations are available with a position in..... Tech...
WHAT????!!!!
It's a common misconception that to work in tech means spending the days chained inside a cube
inside a building- FOREVER!
NOPE... this couldn't
be farther from the truth.
Creates stylish and functional websites
Write applications that are designed for all mobile devices
Writes the code for eqiupment handling websites
Handles both front and back end duties.
Make sure the hardware is running smoothly.
Write programs and apps for all devices
The tech world is open to ANYONE and EVERYONE!
Just start and have fun.


{% extends 'layout.html' %} {% block body %}
You are logged in as admin. Change the password though updating the 'ADMIN_PWD' environment variable.


{% extends "tutorial.html" %} {% block pagebreadcrumb %}{{ tut.title }}{% endblock %} {% block head %} {% endblock %} {% block iscompatible %} var video = document.createElement('video'); return !!(window.AudioContext || window.webkitAudioContext || window.MediaSource); {% endblock %} {% block html5badge %} {% endblock %} {% block content %}
{% include "warning.html" %}
In this article, I'll cover two techniques for streaming audio/video using a
few of the newer multimedia capabilities in web platform. The first method is the
MediaSource API, an API that allows JavaScript to dynamically construct and append media segments to an existing <audio>
or <video>
element.
The second is the collaboration of binary WebSocket and
the Web Audio API to send, reconstruct, and schedule
audio chunks at precise times, producing seamless playback.
An API designed with streaming in mind is the MediaSource API.
It's an experimental feature that allows JavaScript to dynamically append media
to an HTMLMediaElement
.
HTML <audio>
and <video>
are frighteningly trivial to use. Set a src
attribute to some URL, and boom, the browser does its thing: loads the media,
decodes it, buffers it, and handles codec issues.
The devil is in the details though. The cost of such simplicity is less flexibility.
First, we're dealing with the entire file for playback. We have little control
over what the browser does with our media after setting the src
. For example,
what if we want to adaptively change the quality of video based on network conditions?
Or, splice in different sections of video from multiple sources? Aw shucks. Can't!
Without multiple media elements and JS hackery, doing anything
more complex than basic playback becomes a week-long chore when using
<audio>
/<video>
.
The MediaSource API is here to solve these issues. With it, we can tell
<audio>
/<video>
to behave differently.
The MediaSource API is still experimental but is enabled by default in Chrome 23, with a vendor prefix:
function hasMediaSource() {
return !!(window.MediaSource || window.WebKitMediaSource);
}
if (hasMediaSource()) {
// Ready to (html5)rock!
} else {
alert("Bummer. Your browser doesn't support the MediaSource API!");
}
To use MediaSource API, start off with your old buddy HTML5 <video>
:
<video controls autoplay></video>
Note: the examples in this section use <video>
, but the same concepts apply to <audio>
.
Next, create a MediaSource
object:
window.MediaSource = window.MediaSource || window.WebKitMediaSource;
var ms = new MediaSource();
The source is going to be the brains behind the <video>
. Instead of setting
its src
to a file, we're going to create a blob: URL as the MediaSource
. This makes the media element feel special
because it knows we're going to do more than just set a file. We're going to feed it
chunks of video data. Yum!
The rest of the setup looks like this:
function onSourceOpen(e) {
// this.readyState === 'open'. Add a source buffer that expects webm chunks.
sourceBuffer = ms.addSourceBuffer('video/webm; codecs="vorbis,vp8"');
....
}
var ms = new MediaSource();
ms.addEventListener('webkitsourceopen', onSourceOpen.bind(ms), false);
var sourceBuffer = null;
// Use MediaSource to supply video data.
var video = document.querySelector('video');
video.src = window.URL.createObjectURL(ms); // blob URL pointing to the MediaSource.
Only the .webm container is supported at this time.
sourceopen
fires after setting the blob URL. Once that happens, the <video>
is ready to accept incoming data. To hold those bytes, you create a new SourceBuffer
in the callback and inform the buffer what mimetype the <video>
should
expect (webm in this case).
Chunks of .webm can now be dynamically appended to the <video>
:
// Append a portion of a webm file.
sourceBuffer.append(webmChunk);
Herein lies the flexibility of the Media Source API. .append()
takes an Uint8Array
typed array of data. The format of that data is what we've
specified in .addSourceBuffer()
. And the thing we're appending is a byte range
of video data, not an entire .webm movie. This is fundamentally different than how
<video>
normally works.
Of course, if you wanted to, you could append an entire .webm movie in one go!
The previous example appended a single chunk of webm to our <video>
. However,
for the purposes of streaming, we need API functionality that lets us continuously
append new video chunks as they come in from the server. Most people
don't have their media split into a bunch of pieces, so there are a couple of ways
to do this.
Using range requests
If your server supports it, you can request portions of a file using the Range
header. Two APIs that support partial resources out of the box are the Google Drive API and the App Engine BlobStore API (via X-AppEngine-BlobRange
).
You can set custom headers on an XHR request using setRequestHeader()
. For instance, here's an example of requesting the first 500 bytes of a file:
var xhr = new XMLHttpRequest();
xhr.open('GET', '/path/to/video.webm', true);
xhr.responseType = 'arraybuffer';
xhr.setRequestHeader('Range', 'bytes=0-500'); // Request first 500 bytes of the video.
xhr.onload = function(e) {
var initializationWebMChunk = new Uint8Array(e.target.result);
sourceBuffer.append(initializationWebMChunk);
}
xhr.send();
The first portion of a webm file is the "initialization chunk". It contains the container's header information and should always be the first segment added. Don't worry. If your videos are constructed correctly, there's nothing special you need to do. Just make sure the initialization chunk is indeed the first one you append.
For subsequent appends, request the appropriate byte range and go to town:
sourceBuffer.append(webMChunk2);
sourceBuffer.append(webMChunk3);
...
Slicing a file
The second way to dice a file is to do things ahead of time on the server. However, for demonstration purposes, we can do the same thing client-side using the File APIs.
For example, here's how to use XHR to request and slice a file into pieces:
var FILENAME = 'test.webm';
var NUM_CHUNKS = 5;
function get(url, callback) {
var xhr = new XMLHttpRequest();
xhr.open('GET', url, true);
xhr.responseType = 'blob';
xhr.onload = function(e) {
if (this.status == 200) {
callback(this.response);
}
};
xhr.send();
}
get(FILENAME, function(file) {
var chunkSize = Math.ceil(file.size / NUM_CHUNKS);
var fileNameParts = FILENAME.split('.');
for (var i = 0; i < NUM_CHUNKS; ++i) {
var startByte = chunkSize * i;
var chunk = file.slice(startByte, startByte + chunkSize, file.type);
var a = document.createElement('a');
a.download = [fileNameParts[0] + i, fileNameParts[1]].join('.');
a.textContent = 'Download chunk ' + i;
a.title = chunk.size + ' byte';
// blob urls created from file parts use original file. See crbug.com/145156.
a.href = window.URL.createObjectURL(chunk);
document.body.appendChild(a);
}
});
The important bits are:
.responseType
is set to "blob"
to inform the server we're
interested in the resource as a file rather than a string.File.slice()
is used to break up the file into NUM_CHUNKS
pieces.blob:
URL using window.URL.createObjectURL()
and
create a downloadable anchor using a[download]
.When there's no more data to append, call .endOfStream()
to indicate you're done.
This also fires the sourceended
event:
ms.endOfStream();
ms.addEventListener('webkitsourceended', function(e) {
// this.readyState === 'ended'
}, false);
Now we have everything needed for adaptive streaming. For that use case, we can detect network changes in JS and append higher/lower quality video chunks based on the connection.
To demonstrate appending video data onto a <video>
, we need a .webm movie that's
split into multiple pieces. Since you probably don't one of these laying around,
I've created the following script to do that for you. It uses file.slice()
to
break up a .webm file into NUM_CHUNKS
pieces.
Select a .webm file:
Here, we're using XHR2 to pull down the entire webm movie. The important bits to note are:
https://github.com/acolwell/mse-tools
TODO: Link to DashPlayer http://downloads.webmproject.org/adaptive-demo/adaptive/dash-player.html
Todo
http://www.smartjava.org/content/record-audio-using-webrtc-chrome-and-speech-recognition-websockets
Todo


Hello, and welcome to my portfolio! My name is Andrew Mickel, and here you'll find out a bit about me, my experiences with work acnd coding, and even get to see some of the projects I've made!
To get started, go ahead and click one of the tabs up above!


Plugin compatibility search criteria
Target parent cores :
Every cores
Selected cores :
Target plugins :
Every plugins
Selected plugins :


|
General
Sample Code |
TrackerBase Caveats
<PARANOID MODE> Also, here's a repeat of the warning in the attributes section: Warning: do NOT attempt to move attributed files from one partition to another from the command line with the mv command. Your files will move, but they'll lose all their attributes, even when going to another BFS volume. Bad bug. Be careful. It's fine to drag them across volumes with Tracker, however. If you intend to use queries as part of your site generation project, there are a few things to be aware of. For one, queries don't care about directories, so if you have other files that match the same criteria but live somewhere else, like in another folder, your project queries will pick those up as well. There are three solutions to this dilemma: 1) delete those files, 2) zip those files into an archive and delete the originals, and 3) move the files to another mounted volume (queries only search the current volume, by default). A future version of TrackerBase will include a reliable way to filter query results automatically. Another important query caveat is the fact that as of R3.2, command-line queries still search the Trash, unlike the GUI query tool. There is no way to turn this off, so you may have to post-process your queries with sed or a similar tool. For example: query "((name=="filename")&&(BEOS:TYPE=="text/trackerbase"))" | sed /Trash/d See the Snippets page for more on sed and the | operator. When editing attributes with Tracker, it's not possible to delete their entire contents. For example, select an attribute, hit Backspace to delete it, and the original attribute will spring back. There are two possible solutions: use Tracker, but use a single space instead of nothing, or remove the attribute via Terminal with rmattr, then add it back in using "" (two quotation marks with no space between them) as the content. There's another problem associated with using queries to get file listings rather than doing directory crawls: the list of files returned by the shell is a list of complete paths, which makes it difficult to create HTML links to those files. The workaround for this problem is described on the Simple page, but that workaround only works for directory crawls. I'm sure the real solution is a regular expression, but I haven't been able to figure out what. Thus, a second workaround was created for this scenario. If you look at the Tip Server script, you'll see a weird path variable toward the top:
Base=/boot/apps/TrackerBase/projects\tipserver InputPath=$Base/data DataPath=\\/boot\\/apps\\/TrackerBase\\/projects\\/tipserver\\/data\\/In other words, DataPath is the same as InputPath, but includes a ridiculous escape sequence for the slashes. This is used later in the script to subtract that path from each line in the query results list. It works, but requires a hacked variable like this for each query in the script that will be used for creating links. Let me know if you know a better way. Finally, I'm still cutting my teeth on the shell, and I know there's plenty in these scripts that could be improved. I welcome all suggestions<EMAIL_ADDRESS>Next: SimpleTest
|


“The men of other countries,” said Spadevil,“are the slaves of pleasure and desire, knowing it. But the men of my country are the slaves of pleasure and desire, not knowing it.”
“And yet that proud pleasure, which rejoices in self-torture, has something noble in it.”
“He who studies himself at all is ignoble. Only by despising soul as well as body can a man enter into true life.” “On what grounds do they reject women?” “Inasmuch as a woman has ideal love, and cannot live for herself. Love for another is pleasure for the loved one, and therefore injurious to him." “A forest of false ideas is waiting for your axe," said Maskull.“But will they allow it?”
“Spadevil knows, Maskull,” said Tydomin,“that be it today or be it tomorrow, love can't be kept out of a land, even by the disciples of Hator.” “Beware of love—beware of emotion!” exclaimed Spadevil.“Love is but pleasure once removed. Think not of pleasing others, but of serving them.” “Forgive me, Spadevil, if I am still feminine.”
“Right has no sex. So long, Tydomin, as you remember that you are a woman, so long you will not enter into divine apathy of soul.” “But where there are no women, there are no children,” said Maskull.“How came there to be all these generations of Hator men?” “Life breeds passion, passion breeds suffering, suffering breeds the yearning for relief from suffering. Men throng to Sant from all parts, in order to have the scars of their souls healed.”
“In place of hatred of pleasure, which all can understand, what simple formula do you offer?” “Iron obedience to duty,” answered Spadevil. “And if they ask 'How far is this consistent with hatred of pleasure?' what will your pronouncement be?”
“I do not answer them, but I answer you, Maskull, who ask the question. Hatred is passion, and all passion springs from the dark fires of self. Do not hate pleasure at all, but pass it by on one side, calm and undisturbed.” “What is the criterion of pleasure? How can we always recognise it, in order to avoid it?”
“Rigidly follow duty, and such questions will not arise.” Later in the afternoon, Tydomin timidly placed her fingers on Spadevil's arm. “Fearful doubts are in my mind,” she said.“This expedition to Sant may turn out badly. I have seen a vision of you, Spadevil, and myself lying dead and covered in blood, but Maskull was not there.”
“We may drop the torch, but it will not be extinguished, and others will raise it.”


AWS::IoTEvents::DetectorModel
The AWS::IoTEvents::DetectorModel resource creates a detector model. You create a detector model (a model of your equipment or process) using states. For each state, you define conditional (Boolean) logic that evaluates the incoming inputs to detect significant events. When an event is detected, it can change the state or trigger custom-built or predefined actions using other AWS services. You can define additional events that trigger actions when entering or exiting a state and, optionally, when a condition is met. For more information, see How to Use AWS IoT Events in the AWS IoT Events Developer Guide.
When you successfully update a detector model (using the AWS IoT Events console, AWS IoT Events API or CLI
commands, or AWS CloudFormation) all detector instances created by the model are reset to their initial
states. (The detector's state
, and the values of any variables and timers are reset.)
When you successfully update a detector model (using the AWS IoT Events console, AWS IoT Events API or CLI commands, or AWS CloudFormation) the version number of the detector model is incremented. (A detector model with version number 1 before the update has version number 2 after the update succeeds.)
If you attempt to update a detector model using AWS CloudFormation and the update does not succeed, the system may, in some cases, restore the original detector model. When this occurs, the detector model's version is incremented twice (for example, from version 1 to version 3) and the detector instances are reset.
Also, be aware that if you attempt to update several detector models at once using AWS CloudFormation, some updates may succeed and others fail. In this case, the effects on each detector model's detector instances and version number depend on whether the update succeeded or failed, with the results as stated.
Syntax
To declare this entity in your AWS CloudFormation template, use the following syntax:
JSON
{ "Type" : "AWS::IoTEvents::DetectorModel", "Properties" : { "DetectorModelDefinition" :
DetectorModelDefinition
, "DetectorModelDescription" :String
, "DetectorModelName" :String
, "EvaluationMethod" :String
, "Key" :String
, "RoleArn" :String
, "Tags" :[ Tag, ... ]
} }
YAML
Type: AWS::IoTEvents::DetectorModel Properties: DetectorModelDefinition:
DetectorModelDefinition
DetectorModelDescription:String
DetectorModelName:String
EvaluationMethod:String
Key:String
RoleArn:String
Tags:- Tag
Properties
DetectorModelDefinition
-
Information that defines how a detector operates.
Required: Yes
Type: DetectorModelDefinition
Update requires: No interruption
DetectorModelDescription
-
A brief description of the detector model.
Required: No
Type: String
Maximum:
128
Update requires: No interruption
DetectorModelName
-
The name of the detector model.
Required: No
Type: String
Minimum:
1
Maximum:
128
Pattern:
^[a-zA-Z0-9_-]+$
Update requires: Replacement
EvaluationMethod
-
Information about the order in which events are evaluated and how actions are executed.
Required: No
Type: String
Allowed values:
BATCH | SERIAL
Update requires: No interruption
Key
-
The value used to identify a detector instance. When a device or system sends input, a new detector instance with a unique key value is created. AWS IoT Events can continue to route input to its corresponding detector instance based on this identifying information.
This parameter uses a JSON-path expression to select the attribute-value pair in the message payload that is used for identification. To route the message to the correct detector instance, the device must send a message payload that contains the same attribute-value.
Required: No
Type: String
Minimum:
1
Maximum:
128
Pattern:
^((`[\w\- ]+`)|([\w\-]+))(\.((`[\w- ]+`)|([\w\-]+)))*$
Update requires: Replacement
RoleArn
-
The ARN of the role that grants permission to AWS IoT Events to perform its operations.
Required: Yes
Type: String
Minimum:
1
Maximum:
2048
Update requires: No interruption
Tags
-
An array of key-value pairs to apply to this resource.
For more information, see Tag.
Required: No
Type: List of Tag
Update requires: No interruption
Return values
Ref
When you pass the logical ID of this resource to the intrinsic Ref
function, Ref
returns the name of the detector model. For example:
{"Ref": "myDetectorModel"}
For the AWS IoT Events detector model myDetectorModel
, Ref
returns the name of the
detector model.
For more information about using the Ref
function, see Ref.
Examples
Simple Detector Model
The following example creates a simple detector model with only one state.
JSON
{ "Description": "Simple Detector Model Template", "Resources": { "MyDetectorModel": { "Type": "AWS::IoTEvents::DetectorModel", "Properties": { "DetectorModelName": "myDetectorModel", "DetectorModelDescription": "My Detector Model created by CloudFormation", "Key": "myKey", "RoleArn": { "Fn::GetAtt" : [ "myRole", "Arn" ] }, "DetectorModelDefinition": { "InitialStateName": "myInitialState", "States": [ { "StateName": "myInitialState", "OnInput": { "Events": [ { "EventName": "onInputPublishEvent", "Condition": { "Fn::Join" : [ ".", ["$input", {"Ref": "myInput"}, "foo > 1"] ] }, "Actions": [ { "IotTopicPublish": { "MqttTopic": "myMqttTopic" } } ] } ] } } ] } } } } }
YAML
--- Description: "Simple Detector Model Template" Resources: MyDetectorModel: Type: "AWS::IoTEvents::DetectorModel" Properties: DetectorModelName: "myDetectorModel" DetectorModelDescription: "My Detector Model created by CloudFormation" Key: "myKey" RoleArn: !GetAtt myRole.Arn DetectorModelDefinition: InitialStateName: "myInitialState" States: - StateName: "myInitialState" OnInput: Events: - EventName: "onInputPublishEvent" Condition: !Join [".", ["$input", {'Ref': myInput}, "foo > 1"]] Actions: - IotTopicPublish: MqttTopic: "myMqttTopic"
Full Detector Model
The following example creates a more complete example of a detector model with two states.
JSON
{ "Description": "Detector Model Template for CloudFormation", "Resources": { "MyDetectorModel": { "Type": "AWS::IoTEvents::DetectorModel", "Properties": { "DetectorModelName": "myDetectorModel", "DetectorModelDescription": "My Detector Model created by CloudFormation", "Key": "myKey", "RoleArn": "arn:aws:iam::123456789012:role/myIotEventsRole", "DetectorModelDefinition": { "InitialStateName": "myInitialState", "States": [ { "StateName": "myInitialState", "OnEnter": { "Events": [ { "EventName": "onEnterEvent", "Actions": [ { "SetVariable": { "VariableName": "Variable", "Value": "0" } } ] }, { "EventName": "onEnter Event 2", "Condition": "true", "Actions": [ { "SetTimer": { "TimerName": "myTimer", "Seconds": 60 } } ] } ] }, "OnInput": { "Events": [ { "EventName": "onInputEvent", "Condition": { "Fn::Join" : [ ".", ["$input", {"Ref": "myInput"}, "foo > 1"] ] }, "Actions": [ { "IotTopicPublish": { "MqttTopic": "myMqttTopic" } }, { "ResetTimer": { "TimerName": "myTimer" } } ] } ], "TransitionEvents": [ { "EventName": "Transit to other state", "Condition": "true", "Actions": [ { "Sns": { "TargetArn": "arn:aws:sns:123456789012:mySnsTopic" } }, { "Lambda": { "FunctionArn": "arn:aws:lambda:123456789012:function:myLambdaFunction" } }, { "Firehose": { "DeliveryStreamName": "myStreamName", "Separator": "," } }, { "Sqs": { "QueueUrl": "myQueueUrl", "UseBase64": true } }, { "IotEvents": { "InputName": "myInputName" } } ], "NextState": "myOtherState" } ] }, "OnExit": { "Events": [ { "EventName": "Clear timers", "Condition": "1 == 1", "Actions": [ { "ClearTimer": { "TimerName": "myTimer" } } ] } ] } }, { "StateName": "myOtherState", "OnEnter": { "Events": [ { "EventName": "onEnterEvent", "Actions": [ { "SetVariable": { "VariableName": "Variable", "Value": "0" } } ] }, { "EventName": "onEnter Event 2", "Condition": "true", "Actions": [ { "SetTimer": { "TimerName": "myTimer", "Seconds": 60 } } ] } ] }, "OnExit": { "Events": [ { "EventName": "Clear timers", "Condition": "1 == 1", "Actions": [ { "ClearTimer": { "TimerName": "myTimer" } } ] } ] } } ] } } } } }
YAML
--- Description: "Detector Model Template for CloudFormation" Resources: MyDetectorModel: Type: "AWS::IoTEvents::DetectorModel" Properties: DetectorModelName: "myDetectorModel" DetectorModelDescription: "My Detector Model created by CloudFormation" Key: "myKey" RoleArn: "arn:aws:iam::123456789012:role/myIotEventsRole" DetectorModelDefinition: InitialStateName: "myInitialState" States: - StateName: "myInitialState" OnEnter: Events: - EventName: "onEnterEvent" Actions: - SetVariable: VariableName: "Variable" Value: "0" - EventName: "onEnter Event 2" Condition: "true" Actions: - SetTimer: TimerName: "myTimer" Seconds: 60 OnInput: Events: - EventName: "onInputEvent" Condition: !Join [".", ["$input", {'Ref': myInput}, "foo > 1"]] Actions: - IotTopicPublish: MqttTopic: "myMqttTopic" - ResetTimer: TimerName: "myTimer" TransitionEvents: - EventName: "Transit to other state" Condition: "true" Actions: - Sns: TargetArn: "arn:aws:sns:123456789012:mySnsTopic" - Lambda: FunctionArn: "arn:aws:lambda:123456789012:function:myLambdaFunction" - Firehose: DeliveryStreamName: "myStreamName" Separator: "," - Sqs: QueueUrl: "myQueueUrl" UseBase64: true - IotEvents: InputName: "myInputName" NextState: "myOtherState" OnExit: Events: - EventName: "Clear timers" Condition: "1 == 1" Actions: - ClearTimer: TimerName: "myTimer" - StateName: "myOtherState" OnEnter: Events: - EventName: "onEnterEvent" Actions: - SetVariable: VariableName: "Variable" Value: "0" - EventName: "onEnter Event 2" Condition: "true" Actions: - SetTimer: TimerName: "myTimer" Seconds: 60 OnExit: Events: - EventName: "Clear timers" Condition: "1 == 1" Actions: - ClearTimer: TimerName: "myTimer"
See also
-
How to Use AWS IoT Events in the AWS IoT Events Developer Guide
-
CreateDetectorModel in the AWS IoT Events API Reference


|
06.22.99
Dear Eric Ayzenberg: I enjoyed speaking with Esteban last Friday, but regret that I didn't get an opportunity to talk to you about the design position you had available and to show you some of my work. I was looking forward to meeting you because I was very impressed with what I learned about your group from the website. I went to New York over the weekend and had a chance to visit the Guggenheim while I was there. (I regret not being able to chat with you about it!) It was amazing, overwhelming. I am more impressed with Frank Lloyd Wright's work than I ever was before. I notice a definite similarity between yours and Wright's philosophy of design----that every element in the design matters----for Wright, to render the most aesthetic and useful living experience; for you and your company, to help your clients make the most effective impact on their customers. I respect that approach very much. Esteban mentioned that one of the first responsibilities of your new hire would be to help tackle a backlog of work that you have. If there is anything I can do to help, I would consider it a pleasure to do contract/freelance work for you. Please do keep me in mind. |
|
|
Sincerely, Beatrice Aispuro |
|
| b e g r a f i c . c o m |


Here you will learn a little about me, my interests and some projects that I have been working on.
Learn MoreNice to meet you, I'm Grant. A Wisconsin native, I've been in the Bay Area for 13 years working in the high technology industry in a variety of roles including analytics, ecommerce and business strategy. On the personal front, I am an avid basketball player and passionate about coaching youth basketball. I am married to my wife of 3 years, and we have an awesome dog who keeps us busy.
13+ Years


|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object roboguice.util.SafeAsyncTask<ResultT>
ResultT
- public abstract class SafeAsyncTask<ResultT>
A class similar but unrelated to android's AsyncTask
.
Unlike AsyncTask, this class properly propagates exceptions.
If you're familiar with AsyncTask and are looking for AsyncTask.doInBackground(Object[])
,
we've named it Callable.call()
here to conform with java 1.5's Callable
interface.
Current limitations: does not yet handle progress, although it shouldn't be
hard to add.
If using your own executor, you must call future() to get a runnable you can execute.
| Nested Class Summary | |
|---|---|
static class |
SafeAsyncTask.Task<ResultT>
|
| Field Summary | |
|---|---|
protected static java.util.concurrent.Executor |
DEFAULT_EXECUTOR
|
static int |
DEFAULT_POOL_SIZE
|
protected java.util.concurrent.Executor |
executor
|
protected java.util.concurrent.FutureTask<java.lang.Void> |
future
|
protected android.os.Handler |
handler
|
protected java.lang.StackTraceElement[] |
launchLocation
|
| Constructor Summary | |
|---|---|
SafeAsyncTask()
Sets executor to Executors.newFixedThreadPool(DEFAULT_POOL_SIZE) and Handler to new Handler() |
|
SafeAsyncTask(java.util.concurrent.Executor executor)
Sets Handler to new Handler() |
|
SafeAsyncTask(android.os.Handler handler)
Sets executor to Executors.newFixedThreadPool(DEFAULT_POOL_SIZE) |
|
SafeAsyncTask(android.os.Handler handler,
java.util.concurrent.Executor executor)
|
| Method Summary | |
|---|---|
boolean |
cancel(boolean mayInterruptIfRunning)
|
void |
execute()
|
protected void |
execute(java.lang.StackTraceElement[] launchLocation)
|
java.util.concurrent.Executor |
executor()
|
SafeAsyncTask<ResultT> |
executor(java.util.concurrent.Executor executor)
|
java.util.concurrent.FutureTask<java.lang.Void> |
future()
|
android.os.Handler |
handler()
|
SafeAsyncTask<ResultT> |
handler(android.os.Handler handler)
|
protected SafeAsyncTask.Task<ResultT> |
newTask()
|
protected void |
onException(java.lang.Exception e)
Logs the exception as an Error by default, but this method may be overridden by subclasses. |
protected void |
onFinally()
|
protected void |
onInterrupted(java.lang.InterruptedException e)
Called when the thread has been interrupted, likely because the task was canceled. |
protected void |
onPreExecute()
|
protected void |
onSuccess(ResultT t)
|
| Methods inherited from class java.lang.Object |
|---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Methods inherited from interface java.util.concurrent.Callable |
|---|
call |
| Field Detail |
|---|
public static final int DEFAULT_POOL_SIZE
protected static final java.util.concurrent.Executor DEFAULT_EXECUTOR
protected android.os.Handler handler
protected java.util.concurrent.Executor executor
protected java.lang.StackTraceElement[] launchLocation
protected java.util.concurrent.FutureTask<java.lang.Void> future
| Constructor Detail |
|---|
public SafeAsyncTask()
public SafeAsyncTask(android.os.Handler handler)
public SafeAsyncTask(java.util.concurrent.Executor executor)
public SafeAsyncTask(android.os.Handler handler, java.util.concurrent.Executor executor)
| Method Detail |
|---|
public java.util.concurrent.FutureTask<java.lang.Void> future()
public SafeAsyncTask<ResultT> executor(java.util.concurrent.Executor executor)
public java.util.concurrent.Executor executor()
public SafeAsyncTask<ResultT> handler(android.os.Handler handler)
public android.os.Handler handler()
public void execute()
protected void execute(java.lang.StackTraceElement[] launchLocation)
public boolean cancel(boolean mayInterruptIfRunning)
protected void onPreExecute() throws java.lang.Exception
Exception,
- captured on passed to onException() if present.
java.lang.Exception
protected void onSuccess(ResultT t) throws java.lang.Exception
t
- the result of Callable.call()
Exception,
- captured on passed to onException() if present.
java.lang.Exception
protected void onInterrupted(java.lang.InterruptedException e)
onException(Exception)
, but this method
may be overridden to handle interruptions differently than other
exceptions.
e
- the exception thrown from onPreExecute()
, Callable.call()
, or onSuccess(Object)
protected void onException(java.lang.Exception e) throws java.lang.RuntimeException
e
- the exception thrown from onPreExecute()
, Callable.call()
, or onSuccess(Object)
RuntimeException,
- ignored
java.lang.RuntimeException
protected void onFinally() throws java.lang.RuntimeException
RuntimeException,
- ignored
java.lang.RuntimeException
protected SafeAsyncTask.Task<ResultT> newTask()
|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |


shooter-rust-wasm
Game Over
Restart
Paused
Frame: 1
Score: 0
Kills: 0
Power: 0
Wave: 0
Shots: 0
Weapon: 0
Source on
GitHub
.


Regular Expressions, also known as regexp, is use to determine whether a specific string contains a given pattern. Regexp is usually distinguishable by the forward slashes (/). In the example below, we will try to locate the pattern 'peanut' within the string 'organic peanut butter'.
/peanut/ =~ 'organic peanut butter' returns 8
By using =~, Ruby returns the index position of the first match that is found within the string. In the case of 'organic peanut butter', peanut starts at index position 8. If a match is not located, Ruby will return nil.
/peanut/.match('organic peanut butter') returns MatchData "peanut"
.match will return a MatchData followed by what was matched. In our case, Ruby returns the object, peanut. Similiar to =~, if no match is found, Ruby will return nil.
Ruby classifies the following as metacharacters: ( ) [ ] { } ? + * . In order to use metacharacters in regexp, back slashes are needed.
/peanut\+butter/.match('organic+peanut+butter') returns MatchData "peanut+butter"
Square bracket ([ ]) are used within regexp for character class. Character class tells regexp that it has to match with only one of the characters listed. For example, [ae] tells Ruby to look in the string for either 'a' OR 'e'. In contrast, /ae/ tells Ruby that in order for a match to occurred,'a' must exist followed by 'e'.
/[ae]/ =~ 'cat' returns 1 /ae/ =~ 'cat' returns nil
In the example below, you can find a hyphen (-) in between the square brackets. The hyphen denotes a range, so [a-d] is the equivalent of [abcd].
/[a-d]/ =~ 'robot' returns 2 /[abcd]/ =~ 'robot' returns 2


@(userForm: Form[User])
@maintemplate("Triple X Archery: Digital Scoring Notebook") {
Archers
Record and access your scores via desktop or mobile device.
Network
See the accomplishments of your friends via an activity feed.
Data
Watch your improvements and trends plotted over time.

}


KATHARSIS
Katharsis/Catharsis is to release your repressed emotions and experiences.
Here is a place to share the hardest thing you ever had to go through, how you were feeling about it at that time, and how you dealt/are dealing with it. It will be not only a release of your emotions but also a hint for those who have the same
situation.
【 rule 】
1. Do not share anything that others can identify individuals.
2. You don't need to share your name.
3. 400 characters maximum.


11. Layers
11.1. Client Layer
There are various technical approaches to building GUI clients. The devonfw proposes rich clients that connect to the server via data-oriented services (e.g. using REST with JSON). In general, we have to distinguish among the following types of clients:
-
web clients
-
native desktop clients
-
(native) mobile clients
Our main focus is on web-clients. In our sample application my-thai-star we offer a responsive web-client based on Angular following devon4ng that integrates seamlessly with the back ends of my-thai-star available for Java using devon4j as well as .NET/C# using devon4net. For building angular clients read the separate devon4ng guide.
11.1.1. JavaScript for Java Developers
In order to get started with client development as a Java developer we give you some hints to get started. Also if you are an experienced JavaScript developer and want to learn Java this can be helpful. First, you need to understand that the JavaScript ecosystem is as large as the Java ecosystem and developing a modern web client requires a lot of knowledge. The following table helps you as experienced developer to get an overview of the tools, configuration-files, and other related aspects from the new world to learn. Also it helps you to map concepts between the ecosystems. Please note that we list the tools recommended by devonfw here (and we know that there are alternatives not listed here such as gradle, grunt, bower, etc.).
| Topic | Aspect | JavaScript | Java |
|---|---|---|---|
Programming |
Language |
TypeScript (extends JavaScript) |
|
Runtime |
VM |
nodejs (or web-browser) |
|
Build- & Dependency-Management |
Tool |
||
Config |
|||
Repository |
|||
Build cmd |
|
||
Test cmd |
|
mvn test |
|
Test-Tool |
|||
Test-Runner |
|||
E2E Testing |
|||
Code Analysis |
Code Coverage |
||
Development |
IDE |
||
Framework |
Angular (etc.) |
11.2. Service Layer
The service layer is responsible for exposing functionality made available by the logical layer to external consumers over a network via technical protocols.
11.2.1. Types of Services
Before you start creating your services you should consider some general design aspects:
-
Do you want to create a RPC service?
-
Or is your problem better addressed by messaging or eventing?
-
Who will consume your service?
-
Do you have one or multiple consumers?
-
Do web-browsers have to use your service?
-
Will apps from other vendors or parties have to consume your service that you can not influence if the service may have to change or be extended?
-
11.2.2. Versioning
For RPC services consumed by other applications we use versioning to prevent incompatibilities between applications when deploying updates. This is done by the following conventions:
-
We define a version number and prefix it with
v
(e.g.v1
). -
If we support previous versions we use that version numbers as part of the Java package defining the service API (e.g.
com.foo.application.component.service.api.v1
) -
We use the version number as part of the service name in the remote URL (e.g.
https://application.foo.com/services/rest/component/v1/resource
) -
Whenever breaking changes are made to the API, create a separate version of the service and increment the version (e.g.
v1
→v2
) . The implementations of the different versions of the service contain compatibility code and delegate to the same unversioned use-case of the logic layer whenever possible. -
For maintenance and simplicity, avoid keeping more than one previous version.
11.2.3. Interoperability
For services that are consumed by clients with different technology, interoperability is required. This is addressed by selecting the right protocol, following protocol-specific best practices and following our considerations especially simplicity.
11.2.4. Service Considerations
The term service is quite generic and therefore easily misunderstood. It is a unit exposing coherent functionality via a well-defined interface over a network. For the design of a service, we consider the following aspects:
-
self-contained
The entire API of the service shall be self-contained and have no dependencies on other parts of the application (other services, implementations, etc.). -
idempotence
E.g. creation of the same master-data entity has no effect (no error) -
loosely coupled
Service consumers have minimum knowledge and dependencies on the service provider. -
normalized
Complete, no redundancy, minimal -
coarse-grained
Service provides rather large operations (save entire entity or set of entities rather than individual attributes) -
atomic
Process individual entities (for processing large sets of data, use a batch instead of a service) -
simplicity
Avoid polymorphism, RPC methods with unique name per signature and no overloading, avoid attachments (consider separate download service), etc.
11.2.5. Security
Your services are the major entry point to your application. Hence, security considerations are important here.
See REST Security.
11.3. Service-Versioning
This guide describes the aspect and details about versioning of services
11.3.1. Motivation
Why versioning of services? First of all, you should only care about this topic if you really have to. Service versioning is complex and requires effort (time and budget). The best way to avoid this is to be smart in the first place when designing the service API. Further, if you are creating services where the only consumer is e.g. the web-client that you deploy together with the consumed services then you can change your service without the overhead to create new service versions and keeping old service versions for compatibility.
However, if the following indicators are given you typically need to do service versioning:
-
Your service is part of a complex and distributed IT landscape
-
Your service requires incompatible changes
-
There are many consumers or there is at least one (relevant) consumer that can not be updated at the same time or is entirely out of control (unknown or totally different party/company)
What are incompatible changes?
-
Almost any change when SOAP is used (as it changes the WSDL and breaks the contract). Therefore, we recommend to use REST instead. Then, only the following changes are critical.
-
A change where existing properties (attributes) have to change their name
-
A change where existing features (properties, operations, etc.) have to change their semantics (meaning)
What changes do not cause incompatibilities?
-
Adding new service operations is entirely uncritical with REST.
-
Adding new properties is only a problem in the following cases:
-
Adding new mandatory properties to the input of a service is causing incompatibilities. This problem can be avoided by contract-design.
-
If a consumer is using a service to read data, modify it and then save it back via a service and a property is added to the data, then this property might be lost. This is not a problem with dynamic languages such as JavaScript/TypeScript but with strictly typed languages such as Java. In Java you will typically use structured typed transfer-objects (and not
Map<String, Object>
) so new properties that have been added but are not known to the consumer can not be mapped to the transfer-object and will be lost. When saving that transfer-object later the property will be gone. It might be impossible to determine the difference between a lost property and a property that was removed on purpose. This is a general problem that you need to be aware of and that you have to consider by your design in such situations.
-
Even if you hit an indicator for incompatible changes you can still think about adding a new service operation instead of changing an existing one (and deprecating the old one). Be creative to simplify and avoid extra effort.
11.3.2. Procedure
The procedure when rolling out incompatible changes is illustrated by the following example:
+------+ +------+
| App1 | | App2 |
+---+--+ +--+---+
| |
+---+----+
|
+-------+--------+
| Sv1 |
| |
| App3 |
+----------------+
So, here we see a simple example where App3
provides a Service S
in Version v1
that is consumed both by App1
and App2
.
Now for some reason the service S
has to be changed in an incompatible way to make it future-proof for demands. However, upgrading all 3 applications at the same time is not possible in this case for whatever reason. Therefore, service versioning is applied for the changes of S
.
+------+ +------+
| App1 | | App2 |
+---+--+ +--+---+
| |
+--------+
|
+---+------------+
| Sv1 | Sv2 |
| |
| App3* |
+----------------+
Now, App3
has been upgraded and the new release was deployed. A new version v2
of S
has been added while v1
is still kept for compatibility reasons and that version is still used by App1
and App2
.
+------+ +------+
| App1 | | App2*|
+---+--+ +--+---+
| |
| |
| |
+---+--------+---+
| Sv1 | Sv2 |
| |
| App3 |
+----------------+
Now, App2
has been updated and deployed and it is using the new version v2
of S
.
+------+ +------+
| App1*| | App2 |
+---+--+ +--+---+
| |
+--------+
|
+------------+---+
| Sv1 | Sv2 |
| |
| App3 |
+----------------+
Now, also App1
has been updated and deployed and it is using the new version v2
of S
. The version v1
of S
is not used anymore. This can be verified via logging and monitoring.
+------+ +------+
| App1 | | App2 |
+---+--+ +--+---+
| |
+--------+
|
+------------+---+
| Sv2 |
| |
| App3* |
+----------------+
Finally, version v1
of the service S
was removed from App3
and the new release has been deployed.
11.3.3. Versioning Schema
In general anything can be used to differentiate versions of a service. Possibilities are:
-
Code names (e.g.
Strawberry
,Blueberry
,Grapefruit
) -
Timestamps (
YYYYMMDD-HHmmSS
) -
Sequential version numbers (e.g.
v1
,v2
,v3
) -
Composed version numbers (e.g.
1.0.48-pre-alpha-3-20171231-235959-Strawberry
)
As we are following the KISS principle (see key principles) we propose to use sequential version numbers. These are short, clear, and easy while still allowing to see what version is after another one. Especially composed version numbers (even 1.1
vs. 2.0
) lead to decisions and discussions that easily waste more time than adding value. It is still very easy to maintain an Excel sheet or release-notes document that is explaining the changes for each version (v1
, v2
, v3
) of a particular service.
We suggest to always add the version schema to the service URL to be prepared for service versioning even if service versioning is not (yet) actively used. For simplicity it is explicitly stated that you may even do incompatible changes to the current version (typically v1
) of your service if you can update the according consumers within the same deployment.
11.3.4. Practice
So assuming you know that you have to do service versioning, the question is how to do it practically in the code. The approach for your devon4j project in case of code-first should be as described below:
-
Determine which types in the code need to be changed. It is likely to be the API and implementation of the according service but it may also impact transfer objects and potentially even datatypes.
-
Create new packages for all these concerned types containing the current version number (e.g.
v1
). -
Copy all these types to that new packages.
-
Rename these copies so they carry the version number as suffix (e.g.
V1
). -
Increase the version of the service in the unversioned package (e.g. from
v1
tov2
). -
Now you have two versions of the same service (e.g.
v1
andv2
) but so far they behave exactly the same. -
You start with your actual changes and modify the original files that have been copied before.
-
You will also ensure the links (import statements) of the copied types point to the copies with the version number
-
This will cause incompatibilities (and compile errors) in the copied service. Therefore, you need to fix that service implementation to map from the old API to the new API and behavior. In some cases, this may be easy (e.g. mapping
x.y.z.v1.FooTo
tox.y.z.FooTo
using bean-mapping with some custom mapping for the incompatible changes), in other cases this can get very complex. Be aware of this complexity from the start before you make your decision about service versioning. -
As far as possible this mapping should be done in the service-layer, not to pollute your business code in the core-layer with versioning-aspects. If there is no way to handle it in the service layer, e.g. you need some data from the persistence-layer, implement the "mapping" in the core-layer then, but don’t forget to remove this code, when removing the old service version.
-
Finally, ensure that both the old service behaves as before as well as the new service works as planned.
Modularization
For modularization, we also follow the KISS principle (see key principles):
we suggest to have one api
module per application that will contain the most recent version of your service and get released with every release-version of the application. The compatibility code with the versioned packages will be added to the core
module and therefore is not exposed via the api
module (because it has already been exposed in the previous release of the app). This way, you can always determine for sure which version of a service is used by another application just by its maven dependencies.
The KISS approach with only a single module that may contain multiple services (e.g. one for each business component) will cause problems when you want to have mixed usages of service versions: You can not use an old version of one service and a new version of another service from the same APP as then you would need to have its API module twice as a dependency on different versions, which is not possible. However, to avoid complicated overhead we always suggest to follow this easy approach. Only if you come to the point that you really need this complexity you can still solve it (even afterwards by publishing another maven artefact). As we are all on our way to build more but smaller applications (SOA, microservices, etc.) we should always start simple and only add complexity when really needed.
The following example gives an idea of the structure:
/«my-app»
├──/api
| └──/src/main/java/
| └──/«rootpackage»/«application»/«component»
| ├──/common/api/to
| | └──FooTo
| └──/service/api/rest
| └──FooRestService
└──/core
└──/src/main/java/
└──«rootpackage»/«application»/«component»
├──/common/api/to/v1
| └──FooToV1
└──/service
├──/api/rest/v1
| └──FooRestServiceV1
└──impl/rest
├──/v1
| └── FooRestServiceImplV1
└──FooRestServiceImpl
11.4. Logic Layer
The logic layer is the heart of the application and contains the main business logic. According to our business architecture, we divide an application into components. For each component, the logic layer defines a component-facade. According to the complexity, you can further divide this into individual use-cases. It is very important that you follow the links to understand the concept of component-facade and use-case in order to properly implement your business logic.
11.4.1. Responsibility
The logic layer is responsible to implement the business logic according to the specified functional demands and requirements. Therefore, it creates the actual value of the application. The logic layer is responsible for invoking business logic in external systems. The following additional aspects are also included in its responsibility:
-
transaction-handling (in addition to service layer).
11.4.2. Security
The logic layer is the heart of the application. It is also responsible for authorization and hence security is important in this current case. Every method exposed in an interface needs to be annotated with an authorization check, stating what role(s) a caller must provide in order to be allowed to make the call. The authorization concept is described here.
Direct Object References
A security threat are Insecure Direct Object References. This simply gives you two options:
-
avoid direct object references
-
ensure that direct object references are secure
Especially when using REST, direct object references via technical IDs are common sense. This implies that you have a proper authorization in place. This is especially tricky when your authorization does not only rely on the type of the data and according to static permissions but also on the data itself. Vulnerabilities for this threat can easily happen by design flaws and inadvertence. Here is an example from our sample application:
We have a generic use-case to manage BLOBs. In the first place, it makes sense to write a generic REST service to load and save these BLOBs. However, the permission to read or even update such BLOB depends on the business object hosting the BLOB. Therefore, such a generic REST service would open the door for this OWASP A4 vulnerability. To solve this in a secure way, you need individual services for each hosting business object to manage the linked BLOB and have to check permissions based on the parent business object. In this example the ID of the BLOB would be the direct object reference and the ID of the business object (and a BLOB property indicator) would be the indirect object reference.
11.4.3. Component Facade
For each component of the application, the logic layer defines a component facade.
This is an interface defining all business operations of the component.
It carries the name of the component («Component»
) and has an implementation named «Component»Impl
(see implementation).
API
The component facade interface defines the logic API of the component and has to be business oriented.
This means that all parameters and return types of all methods from this API have to be business transfer-objects, datatypes (String
, Integer
, MyCustomerNumber
, etc.), or collections of these.
The API may also only access objects of other business components listed in the (transitive) dependencies of the business-architecture.
Here is an example how such an API may look like:
public interface Bookingmanagement {
BookingEto findBooking(Long id);
BookingCto findBookingCto(Long id);
Page<BookingEto> findBookingEtos(BookingSearchCriteriaTo criteria);
void approveBooking(BookingEto booking);
}
Implementation
The implementation of an interface from the logic layer (a component facade or a use-case) carries the name of that interface with the suffix Impl
and is annotated with @Named
.
An implementation typically needs access to the persistent data.
This is done by injecting the corresponding repository (or DAO).
According to data-sovereignty, only repositories of the same business component may be accessed directly.
For accessing data from other components the implementation has to use the corresponding API of the logic layer (the component facade). Further, it shall not expose persistent entities from the dataaccess layer and has to map them to transfer objects using the bean-mapper.
@Named
@Transactional
public class BookingmanagementImpl extends AbstractComponentFacade implements Bookingmanagement {
@Inject
private BookingRepository bookingRepository;
@Override
public BookingEto findBooking(Long id) {
LOG.debug("Get Booking with id {} from database.", id);
BookingEntity entity = this.bookingRepository.findOne(id);
return getBeanMapper().map(entity, BookingEto.class));
}
}
As you can see, entities (BookingEntity
) are mapped to corresponding ETOs (BookingEto
).
Further details about this can be found in bean-mapping.
For complex applications, the component facade consisting of many different methods. For better maintainability in such case it is recommended to split it into separate use-cases that are then only aggregated by the component facade.
11.4.4. UseCase
A use-case is a small unit of the logic layer responsible for an operation on a particular entity (business object).
It is defined by an interface (API) with its according implementation.
Following our architecture-mapping, use-cases are named Uc«Operation»«BusinessObject»[Impl]
. The prefix Uc
stands for use-case and allows to easily find and identify them in your IDE. The «Operation»
stands for a verb that is operated on the entity identified by «BusinessObject»
.
For CRUD we use the standard operations Find
and Manage
that can be generated by CobiGen. This also separates read and write operations (e.g. if you want to do CQSR, or to configure read-only transactions for read operations).
Find
The UcFind«BusinessObject»
defines all read operations to retrieve and search the «BusinessObject»
.
Here is an example:
public interface UcFindBooking {
BookingEto findBooking(Long id);
BookingCto findBookingCto(Long id);
Page<BookingEto> findBookingEtos(BookingSearchCriteriaTo criteria);
Page<BookingCto> findBookingCtos(BookingSearchCriteriaTo criteria);
}
Manage
The UcManage«BusinessObject»
defines all CRUD write operations (create, update and delete) for the «BusinessObject»
.
Here is an example:
public interface UcManageBooking {
BookingEto saveBooking(BookingEto booking);
boolean deleteBooking(Long id);
}
Custom
Any other non CRUD operation Uc«Operation»«BusinessObject»
uses any other custom verb for «Operation»
.
Typically, such custom use-cases only define a single method.
Here is an example:
public interface UcApproveBooking {
void approveBooking(BookingEto booking);
}
Implementation
For the implementation of a use-cas, the same rules that are described for the component-facade implementation.
.
However, when following the use-case approach, your component facade simply changes to:
public interface Bookingmanagement extends UcFindBooking, UcManageBooking, UcApproveBooking {
}
Where the implementation only delegates to the use-cases and gets entirely generated by CobiGen:
public class BookingmanagementImpl implements {
@Inject
private UcFindBooking ucFindBooking;
@Inject
private UcManageBooking ucManageBooking;
@Inject
private UcApproveBooking ucApproveBooking;
@Override
public BookingEto findBooking(Long id) {
return this.ucFindBooking.findBooking(id);
}
@Override
public Page<BookingEto> findBookingEtos(BookingSearchCriteriaTo criteria) {
return this.ucFindBooking.findBookingEtos(criteria);
}
@Override
public BookingEto saveBooking(BookingEto booking) {
return this.ucManageBooking.saveBooking(booking);
}
@Override
public boolean deleteBooking(Long id) {
return this.ucManageBooking.deleteBooking(booking);
}
@Override
public void approveBooking(BookingEto booking) {
this.ucApproveBooking.approveBooking(booking);
}
...
}
This approach is also illustrated by the following UML diagram:
Internal use case
Sometimes, a component with multiple related entities and many use-cases needs to reuse business logic internally.
Of course, this can be exposed as an official use-case API but this will imply using transfer-objects (ETOs) instead of entities. In some cases, this is undesired e.g. for better performance to prevent unnecessary mapping of entire collections of entities.
In the first place, you should try to use abstract base implementations providing reusable methods the actual use-case implementations can inherit from.
If your business logic is even more complex and you have multiple aspects of business logic to share and reuse but also run into multi-inheritance issues, you may also just create use-cases that have their interface located in the impl
scope package right next to the implementation (or you may just skip the interface). In such a case, you may define methods that directly take or return entity objects.
To avoid confusion with regular use-cases, we recommend to add the Internal
suffix to the type name leading to Uc«Operation»«BusinessObject»Internal[Impl]
.
Injection issues
Technically, now you have two implementations of your use-case:
-
the direct implementation of the use-case (
Uc*Impl
) -
the component facade implementation (
«Component»Impl
)
When injecting a use-case interface this could cause ambiguities. This is addressed as following:
-
In the component facade implementation (
«Component»Impl
) spring is smart enough to resolve the ambiguity as it assumes that a spring bean never wants to inject itself (it can already be an access viathis
). Therefore, only the proper use-case implementation remains as a candidate and injection works as expected. -
In all other places, simply always inject the component facade interface instead of the use-case.
In case you might have the lucky occasion to hit this nice exception:
org.springframework.beans.factory.BeanCurrentlyInCreationException: Error creating bean with name 'uc...Impl': Bean with name 'uc...Impl' has been injected into other beans [...Impl] in its raw version as part of a circular reference, but has eventually been wrapped. This means that said other beans do not use the final version of the bean. This is often the result of over-eager type matching - consider using 'getBeanNamesOfType' with the 'allowEagerInit' flag turned off, for example.
To get rid of such an error you need to annotate your according implementation also with @Lazy
in addition to @Named
.
11.5. Data-Access Layer
The data-access layer is responsible for all outgoing connections to access and process data. This is mainly about accessing data from a persistent data-store. External system could also be accessed from the data-access layer if they match this definition, e.g. a mongo-db via rest services.
11.5.1. Database
You need to make your choice for a database. Options are documented here.
The classical approach is to use a Relational Database Management System (RDMS). In such a case, we strongly recommend to follow our JPA Guide. Some NoSQL databases are supported by spring-data so you can consider the repository guide.
11.6. Batch Layer
We understand batch processing as a bulk-oriented, non-interactive, typically long running execution of tasks. For simplicity, we use the term "batch" or "batch job" for such tasks in the following documentation.
devonfw uses Spring Batch as a batch framework.
This guide explains how Spring Batch is used in devonfw applications. It focuses on aspects which are special to devonfw. If you want to learn about spring-batch you should adhere to springs references documentation.
There is an example of a simple batch implementation in the my-thai-star batch module.
In this chapter, we will describe the overall architecture (especially concerning layering) and how to administer batches.
11.6.1. Layering
Batches are implemented in the batch layer. The batch layer is responsible for batch processes, whereas the business logic is implemented in the logic layer. Compared to the service layer, you may understand the batch layer just as a different way of accessing the business logic. From a component point of view, each batch is implemented as a subcomponent in the corresponding business component. The business component is defined by the business architecture.
Let’s make an example for that. The sample application implements a batch for exporting ingredients. This ingredientExportJob belongs to the dishmanagement business component. So the ingredientExportJob is implemented in the following package:
<basepackage>.dishmanagement.batch.impl.*
Batches should invoke use cases in the logic layer for doing their work. Only "batch specific" technical aspects should be implemented in the batch layer.
Example: For a batch, which imports product data from a CSV file, this means that all code for actually reading and parsing the CSV input file is implemented in the batch layer. The batch calls the use case "create product" in the logic layer for actually creating the products for each line read from the CSV input file.
Directly accessing data access layer
In practice, it is not always appropriate to create use cases for every bit of work a batch should do. Instead, the data access layer can be used directly. An example for that is a typical batch for data retention which deletes out-of-time data. Often deleting, out-dated data is done by invoking a single SQL statement. It is appropriate to implement that SQL in a Repository or DAO method and call this method directly from the batch. But be careful: this pattern is a simplification which could lead to business logic cluttered in different layers, which reduces the maintainability of your application. It is a typical design decision you have to make when designing your specific batches.
11.6.2. Project structure and packaging
Batches will be implemented in a separate Maven module to keep the application core free of batch dependencies. The batch module includes a dependency on the application core-module to allow the reuse of the use cases, DAOs etc. Additionally the batch module has dependencies on the required spring batch jars:
<dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>mtsj-core</artifactId>
<version>dev-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-batch</artifactId>
</dependency>
</dependencies>
To allow an easy start of the batches from the command line it is advised to create a bootified jar for the batch module by adding the following to the pom.xml
of the batch module:
<build>
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
</resource>
</resources>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<excludes>
<exclude>config/application.properties</exclude>
</excludes>
</configuration>
</plugin>
<!-- Create bootified jar for batch execution via command line.
Your applications spring boot app is used as main-class.
-->
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<mainClass>com.devonfw.application.mtsj.SpringBootApp</mainClass>
<classifier>bootified</classifier>
</configuration>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
11.6.3. Implementation
Most of the details about implementation of batches is described in the spring batch documentation. There is nothing special about implementing batches in devonfw. You will find an easy example in my-thai-star.
11.6.4. Starting from command line
Devonfw advises to start batches via command line. This is most common to many ops teams and allows easy integration in existing schedulers. In general batches are started with the following command:
java -jar <app>-batch-<version>-bootified.jar --spring.main.web-application-type=none --spring.batch.job.enabled=true --spring.batch.job.names=<myJob> <params>
| Parameter | Explanation |
|---|---|
|
This disables the web app (e.g. Tomcat) |
|
This specifies the name of the job to run. If you leave this out ALL jobs will be executed. Which probably does not make to much sense. |
|
(Optional) additional parameters which are passed to your job |
This will launch your normal spring boot app, disables the web application part and runs the designated job via Spring Boots org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner
.
Scheduling
In real world scheduling of batches is not as simple as it first might look like.
-
Multiple batches have to be executed in order to achieve complex tasks. If one of those batches fails the further execution has to be stopped and operations should be notified for example.
-
Input files or those created by batches have to be copied from one node to another.
-
Scheduling batch executing could get complex easily (quarterly jobs, run job on first workday of a month, …)
For devonfw we propose the batches themselves should not mess around with details of scheduling. Likewise your application should not do so. This complexity should be externalized to a dedicated batch administration service or scheduler. This service could be a complex product or a simple tool like cron. We propose Rundeck as an open source job scheduler.
This gives full control to operations to choose the solution which fits best into existing administration procedures.
11.6.5. Handling restarts
If you start a job with the same parameters set after a failed run (BatchStatus.FAILED) a restart will occur. In many cases your batch should then not reprocess all items it processed in the previous runs. For that you need some logic to start at the desired offset. There different ways to implement such logic:
-
Marking processed items in the database in a dedicated column
-
Write all IDs of items to process in a separate table as an initialization step of your batch. You can then delete IDs of already processed items from that table during the batch execution.
-
Storing restart information in springs ExecutionContext (see below)
Using spring batch ExecutionContext for restarts
By implementing the ItemStream
interface in your ItemReader
or ItemWriter
you may store information about the batch progress in the ExecutionContext
. You will find an example for that in the CountJob in My Thai Star.
Additional hint: It is important that bean definition method of your ItemReader
/ItemWriter
return types implementing ItemStream
(and not just ItemReader
or ItemWriter
alone). For that the ItemStreamReader
and ItemStreamWriter
interfaces are provided.
11.6.6. Exit codes
Your batches should create a meaningful exit code to allow reaction to batch errors e.g. in a scheduler.
For that spring batch automatically registers an org.springframework.boot.autoconfigure.batch.JobExecutionExitCodeGenerator
. To make this mechanism work your spring boot app main class as to populate this exit code to the JVM:
@SpringBootApplication
public class SpringBootApp {
public static void main(String[] args) {
if (Arrays.stream(args).anyMatch((String e) -> e.contains("--spring.batch.job.names"))) {
// if executing batch job, explicitly exit jvm to report error code from batch
System.exit(SpringApplication.exit(SpringApplication.run(SpringBootApp.class, args)));
} else {
// normal web application start
SpringApplication.run(SpringBootApp.class, args);
}
}
}
11.6.7. Stop batches and manage batch status
Spring batch uses several database tables to store the status of batch executions. Each execution may have different status. You may use this mechanism to gracefully stop batches. Additionally in some edge cases (batch process crashed) the execution status may be in an undesired state. E.g. the state will be running, despite the process crashed sometime ago. For that cases you have to change the status of the execution in the database.
CLI-Tool
Devonfw provides a easy to use cli-tool to manage the executing status of your jobs.
The tool is implemented in the devonfw module devon4j-batch-tool
. It will provide a runnable jar, which may be used as follows:
- List names of all previous executed jobs
-
java -D'spring.datasource.url=jdbc:h2:~/mts;AUTO_SERVER=TRUE' -jar devon4j-batch-tool.jar jobs list
- Stop job named 'countJob'
-
java -D'spring.datasource.url=jdbc:h2:~/mts;AUTO_SERVER=TRUE' -jar devon4j-batch-tool.jar jobs stop countJob
- Show help
-
java -D'spring.datasource.url=jdbc:h2:~/mts;AUTO_SERVER=TRUE' -jar devon4j-batch-tool.jar
As you can the each invocation includes the JDBC connection string to your database. This means that you have to make sure that the corresponding DB driver is in the classpath (the prepared jar only contains H2).
11.6.8. Authentication
Most business application incorporate authentication and authorization. Your spring boot application will implement some kind of security, e.g. integrated login with username+password or in many cases authentication via an existing IAM. For security reasons your batch should also implement an authentication mechanism and obey the authorization implemented in your application (e.g. via @RolesAllowed).
Since there are many different authentication mechanism we cannot provide an out-of-the-box solution in devonfw, but we describe a pattern how this can be implemented in devonfw batches.
We suggest to implement the authentication in a Spring Batch tasklet, which runs as the first step in your batch. This tasklet will do all of the work which is required to authenticate the batch. A simple example which authenticates the batch "locally" via username and password could be implemented like this:
@Named
public class SimpleAuthenticationTasklet implements Tasklet {
@Override
public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {
String username = chunkContext.getStepContext().getStepExecution().getJobParameters().getString("username");
String password = chunkContext.getStepContext().getStepExecution().getJobParameters().getString("password");
Authentication authentication = new UsernamePasswordAuthenticationToken(username, password);
SecurityContextHolder.getContext().setAuthentication(authentication);
return RepeatStatus.FINISHED;
}
}
The username and password have to be supplied via two cli parameters -username
and -password
. This implementation creates an "authenticated" Authentication
and sets in the Spring Security context. This is just for demonstration normally you should not provide passwords via command line. The actual authentication will be done automatically via Spring Security as in your "normal" application.
If you have a more complex authentication mechanism in your application e.g. via OpenID connect just call this in the tasklet. Naturally you may read authentication parameters (e.g. secrets) from the command line or more securely from a configuration file.
In your Job Configuration set this tasklet as the first step:
@Configuration
@EnableBatchProcessing
public class BookingsExportBatchConfig {
@Inject
private JobBuilderFactory jobBuilderFactory;
@Inject
private StepBuilderFactory stepBuilderFactory;
@Bean
public Job myBatchJob() {
return this.jobBuilderFactory.get("myJob").start(myAuthenticationStep()).next(...).build();
}
@Bean
public Step myAuthenticationStep() {
return this.stepBuilderFactory.get("myAuthenticationStep").tasklet(myAuthenticatonTasklet()).build();
}
@Bean
public Tasklet myAuthenticatonTasklet() {
return new SimpleAuthenticationTasklet();
}
...
11.6.9. Tipps & tricks
Identifying job parameters
Spring uses a jobs parameters to identify job executions. Parameters starting with "-" are not considered for identifying a job execution.


1D is a small demo of a one-dimensional graphical engine. The demo has three components. The first is a first person view of the world as seen in the gif above. Then there is a top down view, mostly for debugging purposes. Finally, there is a level editor to make levels. I developed the engine from scratch in Java as a solo project.
While I like to call this a one-dimensional graphics engine it is really a unique 2D graphical engine. Just like how a 3D engine rasters a 3D world to a 2D viewport, this engine rasters a 2D world to a 1D viewport. In fact, this engine follows the same steps as a 3D engine would except rendering line segments instead of triangles. First, at each frame it calculates a transformation matrix based on where the character is. Then each line is iterated through the rasterization routine. Finally, the frame buffer is drawn to the screen.
The top down view shows the character's position and cone of vision along with the map. The blue square denotes the location of the character, the green lines indicate walls of the map, and the red lines show the character's cone of vision.
The level editor allows for an easier way to make levels. To make a line simply click the two endpoints. Holding shift will make a line either horizontal of vertical. By default, the editor starts a new line every time, so to make connected lines hold control. Backspace will remove the last placed line, whereas delete will delete all lines. Finally, the enter key prints out the level data to console. This data can be saved to a file and then the graphics engine will render it.


Toggle navigation
TheApartmentCompany
Home
About
Contact
Signup
Login
About
About
×
Are you looking for a new house? We have lots of beautiful apartments. Click
here
to find your favorite apartment.


Images:
Search:
Classes:
Bboxes:
Voc folder:
SHORTCUTS:
Mouse WHEEL - zoom in/out image
Mouse RIGHT BUTTON - pan image
Arrows LEFT and RIGHT - cycle images
Arrows Up and DOWN - cycle classes
Key DELETE - remove selected Bbox
Version: 0.2.5 | Copyright © 2018-2019 Draining Sun.