Hi, newbie here. I am interested in the work done in this project for my home energy monitoring needs. I have 3 phase power and hence I am unsure if this solution will work for me. I don't need high accuracy, just proportional accuracy. My objective is to monitor what is using power so I can control it. +/- 5% is probably fine.
From what I have read, here is my understanding,
I can get a single emontx and attach to my 3 phase. I assume this means 1 CT for each phase.
I then can monitor the current only for each phase which will allow the system to provide me information about apparent power only (otherwise I will need 3 x emontx + 3 power outlets to measure real power). I assume the solution will aggregate the total consumption for a total picture.
i see some posts about accuracy of apparent powered vs real power. My question is will a single emontx measuring apparent power likely get me to 95% accuracy? I assume the answer is "it depends".
For or the record, I have normal domestic appliances but it includes 2 reverse cycle air conditioners plus a swimming pool with 3 electric pumps, plus electric hot water.
if I buy 3 x emontx units (one for each phase) I assume I will need 3 power outlets. Do the three outlets need to be specifically connected to each phase (sorry, I don't know if that is a stupid question or not).
thanks for any help
Re: Another newbie 3 phase question
Hi, I'm no expert with 3-phase, but someone will no doubt correct me if get this wrong ...
A single emonTx can only measure voltage on one phase. It can, however, measure current on 3 phases, by using 3 CTs. To calculate the real power on three phases, the sketch would need to extrapolate the voltage signals for the two phases for which voltage is not being measured. This process will introduce some inaccuracy, but may be OK for your needs.
With 6 analogue inputs being available, your best approach may be to use three for voltage and three for current. A PCB is available from another forum member for this purpose. Having populated this board as shown, you would just need to provide 3 CTs and an Arduino or similar. I presume that a suitable sketch will be available from the same source. Otherwise, the 2-phase code that I posted here on 17/10/2013 (for a New Zealand project) could be easily adapted for this purpose.
The above PCB appears to have a single 3-phase input connector (L1, L2, L3 & N). If you can supply this combined feed from a single source, then the only other power outlet you'd need would be for the processor's DC power supply.
For accurate calculation of real power, phase-shift effects should be taken into account. My 'New Zealand' code, as above, includes this feature as I noted at the time of posting:
This code has been optimised for monitoring real power on several channels. PowerCal and phaseCal can be individually set for each CT. I think the Uno platform is well suited for this particular project, but similar code could be run on an emonTx with additional front-end circuitry as required. I hope that it will be of use to anyone who has need of such a solution.
Re: Another newbie 3 phase question
It's interesting how many queries there have been about 3 phase monitoring recently. It's almost unheard of for a domestic property to get 3 phase in the UK, so the emonTX hardware reflects this.
However, in Europe, especially in Scandinavian countries where electric heating is the norm, 3 phase is common. There's an important distinction to be made between domestic and commercial 3-phase usage. In commercial applications, the size of the loads means you can more easily balance the load between the phases, and a significant proportion of load is often made up from 3-phase motors (used for heating, cooling, ventilation, pumping). Domestic properties aren't like this, the phases can be really unbalanced.
For that reason, monitoring the voltage on all three phases can become much more important!
Re: Another newbie 3 phase question
"For that reason, monitoring the voltage on all three phases can become much more important!" Indeed. And don't forget that if the neutral voltage is significant, the angles between the phases will shift too - and it would be almost impossible to compensate for that dynamically. It all gets very interesting.
An alternative to Robin's suggestion: MartinR's "Full Fat 3-phase monitor". It uses 3 stacked emonTx's - Version 2 (V3s could be used but the means of interconnecting them is not so straightforward). That will give you a lot more processing power too.
[I had never come across 3-phase for a domestic supply in the UK before coming to this site, and as far as I know we have two: Noah and MartinR. Noah's is a converted industrial premises, which explains it in his case. I don't know about Martin's.]
To hopefully clarify Robin's answer a little: "To calculate the real power on three phases, the sketch would need to extrapolate the voltage signals for the two phases for which voltage is not being measured. This process will introduce some inaccuracy, but may be OK for your needs." That's not totally accurate. The published 3-phase sketch measures a single voltage and records it, then "plays it back" 120° and 240° later to use it to calculate the real power etc for the 2nd and 3rd phases following on behind. The underlying assumption that leads to a possible inaccuracy is that the 2nd and 3rd phases are exactly the same shape and amplitude as the 1st phase. Hopefully, if all the loads and supply impedances etc are balanced, that will be true. In practice, it won't be, and how far out they will be will depend on your individual circumstances.
Re: Another newbie 3 phase question
Robert,
I'm also going 3 phase. I live on a farm, but don't do much farming and currently have single phase. I have a little bit of 3 phase stuff, compressor, welder and a motor but it was cheaper to just use a generator for that. Our local DNO is replacing the overhead grid around here so I took the opportunity of going 3 phase It still wasn't cheap and I have to have it installed in their maintenance schedule which seems to be stuck!!.
It has 2 main advantages-
1 I can use my 3 phase equipment without running a generator.
2 I can install more solar panels.
Re: Another newbie 3 phase question
A friend (another Tinbum!) who also lives on a farm and doesn't farm has single phase on the same single-phase final transformer as his neighbour who does. Neighbour's milking machine has allegedly blown up a laptop, mains-borne network, a portable tv and another neighbour's central heating. I installed a large surge suppressor for him, so far no more trouble. Except thieves came a month ago in the night and stole his copper overhead feeder. He's on Ali now.
Re: Another newbie 3 phase question
RW: That's not totally accurate. That's not totally accurate. That's not totally accurate.
Sorry Robert, I thought I was making just the same point as you have done. Maybe I should have said that the process of extrapolating, or "playing the voltage samples back later" may introduce some inaccuracy, rather than it necessarily will do. As you've already mentioned, if all three phases are nicely in balance, both approaches should give the same results.
[Edit - But that process is not extrapolation, that is what I was saying. RW]
OK, Chief, so I used the wrong word. Is there a right word for this operation?
Re: Another newbie 3 phase question
When I first moved here 11 years ago I couldn't get my single phase welder to work. It went back under warranty twice but they said it was fine. Some time later I was checking one of my meter on the mains and noticed it read 263v. It turns out that our 5kva !!! transformer was totally on it last legs. It had had quite a few lightening strikes as well they said. It was replaced within 3 hours by the DNO.
They are now going to be putting in a 100 KVa (I'd asked for 50 KVA,the smallest they do). I've also got quite a few surge arresters that I shall be putting in - too many electronics. I'm also thinking of putting them on the DC from the PV panels as well.
We've had quite a few undesirables here as well but thankfully I've spotted them all!!
Re: Another newbie 3 phase question
Power costs in Australia have soared over recent years. They have gone up about 70% in 5 years, and are forecast to keep going up. Given 3-phase is used in Australia (presumably for swimming pools and large air conditioners that are not so common in the UK) then there may be a lot more 3-phase questions coming your way. The problem is that the EmonTx page says you can use an EmonTx for either 4 single phase lines or 1 3-phase, but it doesn't provide any information about 3-phase past that point.
I am no electrician, but I am interested in the Raspberry Pi and also my energy consumption. I thought this project was a great way for me to learn about the RP and also meet my energy interests. So the question now is "what is the best way forward for me?" Cost is not the main factor, but I don't want to get too technical on the power board side. I can buy one or 3 EmonTx devices, and I can clip the CTs where I am told, but not much past that.
1. Is there a way to measure apparent power for 3 phase? Is it just a case of clipping 3 CTs (one per phase)? If so, maybe that would be a good way to start. I could set it up and compare my findings with my power bills and see what happens.
2. Would a retail unit like this http://steplight.com.au/monitor/ew4030-watts-clever-smart-energy-monitor/ work for 3-phase power as an alternative?
3. If I go the whole hog, I assume I will need to do the following (please confirm).
Re: Another newbie 3 phase question
1. If your system voltage is reasonably constant, you can get an indication of apparent power by just measuring the three currents and supplying a fixed voltage (this is how the simplest commercial units operate). Over time and with careful adjustment of the voltage, this can tally reasonably well with your meter. A better way is to measure the voltage of one phase and the currents of three phases. With the "three-phase" sketch, this will give you a good measure of real power in one phase and a good estimate of real power in the other two. Finally, the best way is to actually measure the three voltages and the three currents, which will give you the best accuracy and both real power and apparent power.
2. From a cursory glance at the web page and manual, that unit in essence conveys the reading of your meter into your living room. So it will give you the same information that the flashing LED on your meter gives you - the total energy (from the number of flashes) and power (from the flash rate). The advantage is that by definition it is accurate (unless it misses a pulse), the disadvantage is the information available to you may lack detail - for example you won't know the energy in each phase, and at best it will increment in units of one pulse of your meter - usually but not necessarily 1 Wh.
3. That is certainly one way to go.
Re: Another newbie 3 phase question
Also living in Australia... I had three phases installed in the previous house; 18kW commercial cook top, pool pumps, solar hot water pump, etc.
I moved to the country side a few moths ago... same thing... 3-phase power.
Before living in AU, I lived in Germany... had three phase power... basically all my lif (in numerous places).
My point, it may not be common in the UK, but certainly elsewhere. My critique for OEM was the design is UK focussed and not world focussed, which is what I would expect from an 'Open' system, which implicitly addresses the world. :)
Hi MallyCat, I would be rather looking at this set-up www.ekmmetering.com/ekm-metering-products/electric-meters-kwh-meters.html (rather then Steplight) -- I am currently thinking as to whether I build an OEM with 3 stacked emonTX v2 or get this one. You might consider DIN rail power points in case you are strapped for space in your meterbox.
Yes, power prices will continue to go up given the continued drive for privatisation of public utilities and an energy policy which is ludicrous at best. Which has me also researching the use of a battery bank during night time, rather than exclusive 'dumping' energy to a heater. Thought I understand why it is done in the UK.
Re: Another newbie 3 phase question
MaxG: My critique for OEM was the design is UK focussed and not world focussed, which is what I would expect from an 'Open' system, which implicitly addresses the world. :)
Sure, but the world is a big place, and you have to start somewhere! Its good to see so many contributions now appearing from further afield. The 'Open' in the title is presumably intended to reflect the open-source nature of the enterprise.
"World focussed" sounds like something out of Star Wars :D
Re: Another newbie 3 phase question
An obvious expansion option would be to have "single phase" and "three phase" emonTX designs. One would have three AC inputs, three CT inputs, and however much else there is IO for on a 328, and the other would be the current v3 except it has 1 100A input and 3 30A inputs, rather than the reverse.
Re: Another newbie 3 phase question
The other thing to keep in mind with that pulse-counting solution above is that if you have PV, the meter pulse will only fire when you're importing. I have one of those EM1000 meters shown in the photos in the link above. It typically stops pulsing once the sun comes up and doesn't start again until the sun goes down.
I'm on a very generous feed-in tariff, so monitoring my day time consumption (sourced from the panels) is at least as important as monitoring night time consumption (sourced from the grid).


Runs the random data generator in the LEAN ToolBox to generate realistic market data using a Brownian motion model.
Requires --start <yyyyMMdd>
and --symbol-count <amount>
to be set. The rest of the options have default values.
If --end <yyyyMMdd>
isn't set, data is generated from the start date until the current date.
If the --end
option is set, data is generated between the given --start
and --end
values (inclusive).
By default, dense data is generated, which means the generated data contains at least one data point per resolution step.
You can use --data-density Sparse
to change this to at least one data point per 5 resolution steps, or --data-density VerySparse
to change it to at least one data point per 50 resolution steps.
If the security type is set to Equity
, this command will automatically generate map files, factor files, and coarse universe data. To not generate coarse universe data, set the --include-coarse
option to false
.
The following combinations of security types and resolutions are supported:
| Security Type | Supported Resolutions | ||||
|---|---|---|---|---|---|
| Tick | Second | Minute | Hour | Daily | |
| Equity | |||||
| Forex | |||||
| CFD | |||||
| Future | |||||
| Crypto | |||||
| Option |
By default, the official LEAN engine image is used.
You can override this using the --image <value>
option.
Alternatively, you can set the default engine image for all commands using lean config set engine-image <value>
.
The image is pulled before running the random data generator if it doesn't exist locally yet or if you pass the --update
flag.


Running and Debugging a Plugin
The IntelliJ Platform allows you to run and debug a plugin without leaving the IDE. To run or debug the plugin from within IntelliJ IDEA, you need a configured special profile (a Run/Debug configuration) that specifies the class to run, VM parameters and other specific options. In most cases, you can use the default Run/Debug configuration profiles for your plugin projects.
For information on how to change the Run/Debug configuration profile, refer to Run/Debug Configuration and Run/Debug Configuration: Plugin in IntelliJ IDEA Web Help.
Using IntelliJ IDEA’s debugger, you can find out the origin of the run-time errors and exceptions.
To debug a plugin
- Select Run | Debug in the main menu, or press Shift + F9.
To run a plugin
- Select Run | Run in the main menu, or press Shift + F10.
Last modified:


Ankit Agarwal
Undergraduate Computer Science Major
Some description about me
Projects
TheMovieStreamer
Wingman
Wine-Know


Home
Our Selection Criteria
Action
Another action
Something else here
FAQ
Infos
Action
Another action
Something else here
Sign In
Sign Up
Registration
Registration
Mandatory Information
Personal Data
Company Data
German
English
China
Arabic
Italian
Bangla
You hereby accept the
terms and conditions
Germany
Finland
Estonia
Subscribe Our Newsletter
For Lasted Update
Get Started


I've been coding for just over a year. I am comfortable with JavaScript, HTML, CSS, and Python. I also have some experience with C++ and I have worked with Arduinos.
In my free time, I like to paint and bike ride. I also like to play with my two dogs, Bridget and Daisy. My favorite place to be is the beach.
When I go to college, I would like to major in Computer Science. My top three choices for college at the moment are USC, Cal Poly SLO, and Stevens Institute of Technology. All three schools have excellent CS programs.
At school, I am the co-president of two clubs, Girls Who Code and Asian Culture Club. Both clubs are very important to my identity as a minority in technology and in my community.
Over the summer, I was a TA for Girls Who Code. This job opportunity taught me more about coding and about people in general. The seven weeks I was a TA confirmed my passion for CS and what I want to be once I graduate college.


| {{i+1}}. |
{{productSold.product.name}} |
{{productSold.quantity}} |
{{productSold.sellPriceGross | localizedCurrency}} |
{{productSold.sellPriceNet | localizedCurrency}} |
{{productSold.sellVatValue | localizedCurrency}} |
{{productSold.purchPriceNet | localizedCurrency}} |
{{productSold.purchVatValue | localizedCurrency}} |
0" placement="top" ngbTooltip="{{productSold.previousPeriodQuantity}}">
{{((productSold.quantity-productSold.previousPeriodQuantity)/productSold.previousPeriodQuantity)*100 | number:'1.0-0'}}% |
- |


Alpha-Stable Autoregressive Fast Multichannel Nonnegative Matrix Factorization
for Joint Speech Enhancement and Dereverberation
INTERSPEECH 2021 - A-V-3-2
Mathieu FONTAINE
<EMAIL_ADDRESS>August 31st, 2021
Mathieu FONTAINE , Kouhei SEKIGUCHI, Aditya Arie NUGRAHA, Yoshiaki BANDO, Kazuyoshi YOSHII
I was PhD student in Universite de Lorraine in Nancy on "alpha-stable processes for signal processing" audio source localization, source separation, denoising and restoration
Since October 2019, PostDoc in RIKEN institute "Artificial Intelligence Project" and Guest in Kyoto University in Kazuyoshi Yoshii team "Sound Scene Understanding"
This presentation deals with a technique that reduces the complexity of MNMF in the context of a large family of distribution
Outline
I - Autoregressive FastMNMF $\color{black}\text{[Sek. 20]}$
II - Alpha-Stable Autoregressive FastMNMF $\color{black}\text{[Fon. 21]}$
III - Parameter Estimation
IV - Speech Enhancement and Automatic Speech Transcription Evaluation
V - Conclusion and Future Works
Sekiguchi, K. et al. (2021, ICASSP). Autoregressive Fast Multichannel Nonnegative Matrix Factorization For Joint Blind Source Separation And Dereverberation
Fontaine, M. et al. (2021, INTERSPEECH). Alpha-Stable Autoregressive Fast Multichannel Nonnegative MatrixFactorization for Joint Speech Enhancement and Dereverberation
I - Autoregressive FastMNMF
Why Speech Enhancement and Dereverberation ?
Explain a bit the MBSS and the goal of it
Why Speech Enhancement and Dereverberation ?
Explain a bit the MBSS and the goal of it
Why Speech Enhancement and Dereverberation ?
Explain a bit the MBSS and the goal of it
Why Speech Enhancement and Dereverberation ?
Explain a bit the MBSS and the goal of it
Why Speech Enhancement and Dereverberation ?
Explain a bit the MBSS and the goal of it
Why Speech Enhancement and Dereverberation ?
Explain a bit the MBSS and the goal of it
Autoregressive model
Autoregressive model of order $L$ + STFT representation
$N$ sources capted by $M$ sensors resulting in $T$ time frame and $F$ frequency bins
$$
\bold{x}_{ft} = \bold{d}_{ft} + \bold{r}_{ft} \triangleq \underbrace{\sum_{n=1}^{N}\bold{d}_{nft}}_{{\color{black}\text{direct sound}}} + \underbrace{\sum_{l=\Delta}^{\Delta+L-1} \bold{B}_{fl} \bold{x}_{f,t-l}}_{\color{black}{\text{reverberation}}}
$$
$\Delta:$ Delay
$\bold{B}_{fl}:$ Coefficients of the AR process
Weighted Prediction Error (WPE)$~~\color{black}\text{[Nak. 10]}$
$$
\bold{d}_{ft} \sim \mathcal{N}_{\mathbb{C}}\left(0, \lambda_{ft}\bold{I}\right);
\bold{x}_{ft} \mid \{\bold{x}_{f,t-l}\}_{l=1}^{L} \sim \mathcal{N}_{\mathbb{C}}\left(\bold{r}_{ft}, \lambda_{ft}\bold{I}\right)
$$
Maximization of $\log p \left(\{\bold{x}_{ft}\}_{f,t} \mid \{\bold{B}_{fl}\}_{f,l}, \{\lambda_{ft}\}_{f,t}\right)$ for parameter estimation
Nakatani, T. et al. (2010, ASLP). Speech dereverberation based on variance-normalized delayed linear prediction
Spatial Gaussian Model + MNMF
Local Gaussian Model is the most famous one that admits the reproductive property
NMF is a dictionnary that decomposes the power spectrogram
Duong, N. et al. (2009, TASLP). Under-determined reverberant audio source separation using a full-rank spatial covariance model.
Autoregressive (AR) Fast Gaussian MNMF Models
AR Independent Low-Rank Matrix Analysis (AR-ILRMA) ${\color{black}\text{[Kag. 18}]}$
$\bold{d}_{nft} = \bold{a}_{nf}s_{nft} \quad\quad\quad\quad\quad\quad\quad\quad\quad\quad \text{(Direct sound propagation model)}$
Then $\bold{d}_{nft} \sim \mathcal{N}_{\mathbb{C}}\left({\color{green}\lambda_{nft}}\bold{a}_{nf}\left(\bold{a}_{nf}\right)^{\mathrm{H}}\right)\quad\quad ~~\text{(Rank-1 SCM model)}$
MNMF model for ${\color{green}\lambda_{nft}}$ parameters
AR-Fast MNMF 2: a joint diagonalization (JD) technique${\color{black}\text{ [Sek. 21]}}$
$\bold{d}_{nft} \sim \mathcal{N}_{\mathbb{C}}\left({\color{green}\lambda_{nft}}\underbrace{\bold{Q}_f^{-1}\mathrm{Diag}(\tilde{\bold{g}}_{n})\bold{Q}_f^{-\mathrm{H}}}_{\color{blue}=\bold{G}_{nf}}\right)$
MNMF model for ${\color{green}\lambda_{nft}}$ parameters
AR-ILRMA $\subset$ AR-FastMNMF2
Fast in the sense reduce the number of parameters to estimate
preserve a good performance or improve it
interpretation of FastMNMF2: columns of Q are steering vectors as demixing vector of ILRMA and $g_{nm}$ are weights of rank-1 SCM of the $m^{th}$ direction of source n
Kagami, H. et al. (2018, ICASSP). Joint separation and dereverberation of reverberant mixtures with determined multichannel non-negative matrix factorization
Sekiguchi, K. et al. (2021, ICASSP). Autoregressive Fast Multichannel Nonnegative Matrix Factorization For Joint Blind Source Separation And Dereverberation
AR-FastMNMF2 Optimization
Parameter estimation
$$\tilde{d}_{ftm} = \left|\bold{q}_{fm}^{\mathrm{H}}\bold{d}_{ft}\right|^{2};
\tilde{y}_{ftm} = \sum_{n=1}^{N}\lambda_{nft}\tilde{g}_{nm};
\bold{V}_{fm} = \frac{1}{T}\sum_{t=1}^{T}\bold{d}_{ft}\bold{d}_{ft}^{\mathrm{H}} y_{ftm}^{-1}
$$
Expectation-Maximization approach $\implies$ maximization of the log-likelihood
$w_{nfk}\leftarrow w_{nfk}\sqrt{\frac{\sum_{t, m=1}^{T, M} h_{nkt}
\tilde{g}_{nm} \tilde{d}_{ftm}\tilde{y}_{ftm}^{-2}}
{\sum_{t, m=1}^{T, M} h_{nkt} \tilde{g}_{nm} \tilde{y}_{ftm}^{-1}}}; ~~
h_{nkt}\leftarrow h_{nkt}\sqrt{\frac{\sum_{f, m=1}^{F, M} w_{nfk}
\tilde{g}_{nm} \tilde{d}_{ftm}\tilde{y}_{ftm}^{-2}}
{\sum_{f, m=1}^{F, M} w_{nfk} \tilde{g}_{nm} \tilde{y}_{ftm}^{-1}}};
$
$\tilde{g}_{nm}\leftarrow \tilde{g}_{nm}\sqrt{\frac{\sum_{f, t, m=1}^{F, T, M} \lambda_{nft}
\tilde{d}_{ftm}\tilde{y}_{ftm}^{-2}}
{\sum_{f, t, m=1}^{F, T, M} \lambda_{nft}\tilde{y}_{ftm}^{-1}}}$
$\bold{q}_{fm}\leftarrow(\bold{Q}_f \bold{V}_{fm})^{-1}\bold{e}_m;\quad
\bold{q}_{fm}\leftarrow(\bold{q}_{fm}^{\mathrm{H}}\bold{V}_{fm}\bold{q}_{fm})^{-\frac{1}{2}}\bold{q}_{fm}$ ${\color{black}\text{[Ono 11]}}$
AR coefficients $\bold{B} \triangleq \{\bold{B}_{fl}\}_{f,l=1}^{F,L}$ are updated as in ${\color{black}\text{[Nak. 11]}}$
find a lower-bound of the log-likelihood
equivalent to minimize the IS divergence
non-increase cost function guarantee
N. Ono et al. (WASPAA, 2011). Stable and fast update rules for independent vector analy-sis based on auxiliary function technique.
T. Nakatani et al. (ASLP, 2020). Jointly optimal denoising, dereverberation,and source separation.
II - Alpha-Stable Autoregressive FastMNMF
All the distributions that can be informaly expressed as a Gaussian
Why heavy-tailed models for dereverberation?
Direct sound $\implies$ is an outlier in the time domain compare to the early reflections and the late reverberation
NMF initialization $\implies$ often tricky with a light-tail model${~\color{black}\text{[Bou. 08]}}$
Light tails $\implies$ less robust against impulsive noise or uncommon scenario
heavy-tailed means that asymptotically, the tail of the pdf. are lower than the one of the exponential distribution
MNMF
Boutsidis C. (2008, Pattern Recognition). SVD based initialization: A head start for nonnegative matrix factorization
Why $\alpha$-stable theory ?
Heavy-tailed + stability by linear combination (sum of $\alpha$-stable is $\alpha$-stable)
$\alpha \in (0,2]$: the smaller it is, the heavier are the tails of the distribution
$\alpha=2$: Gaussian, $\alpha=1$: Cauchy, $\alpha=0.5$: Levy
Generalized the multichannel Wiener filter in the fractional domain${~\color{black}\text{[Fon. 20]}}$
Samoradnitsky G. et al. (Chapman, 1994). Non-Gaussian Random Processes: Stochastic Models with Infinite Variance.
Fontaine M. et al. (Signal, 2020). Separation of $\alpha$-stable Random Vectors.
Complex-valued Multivariate Isotropic Symmetric
Elliptically Contoured $\alpha$-stable distribution ($\mathcal{S}_{\mathbb{C}}^{\alpha}(0, \bold{R}))$
Two parameters: $\alpha$ and a positive semidefinite shape matrix $\bold{R}$
$\bold{u}\sim\mathcal{S}_{\mathbb{C}}^{\alpha}$ can be seen as a Gaussian where the covariance is perturbed
The shape matrix model designs a natural AR-FastMNMF extension
rotationnaly invariant
infinite divisible distribution: can be decomposed as a sum of i.i.d. random vector
NIG univariate: stable by linear combination
NIG multivariate: stable only along the shape parameter delta
$\alpha$-AR-FastMNMF
Source TF-independent $\mathcal{S}_{\mathbb{C}}^{\alpha}$ model + ARFastMNMF on direct sources entries $\bold{d}_{nft}$:
$\mathbf{d}_{nft} \mid {\color{red} \phi_{nt}} \sim \mathcal{N}_{\mathbb{C}}\left(\mathbf{0},{\color{red} \phi_{nt}}\bold{Y}_{nft}\right)$
$\bold{Y}_{nft}\triangleq\lambda_{nft}\bold{Q}_f^{-1}\mathrm{Diag}(\tilde{\bold{g}}_{n})\bold{Q}_f^{-\mathsf{H}}$
${\color{red} \bold{\Phi}\triangleq \{\phi_{nt}\}_{n,t=1}^{N,T}}$ only depends on source index and time frame
The mixing model becomes:
$\mathbf{x}_{ft} \mid {\color{red} \bold{\Phi}} \sim \mathcal{N}_{\mathbb{C}}\left(\mathbf{0},\sum_{n=1}^{N}{\color{red}\phi_{nt}}\bold{Y}_{nft}\right)$
Knowing $\bold{\Theta}=\{\{\lambda_{nft}\}, \bold{B},\{\bold{Q}_{f}\}, \{\tilde{\bold{g}}_{n}\}\}, {\color{red} \bold{\Phi}}, \bold{x}_{ft}, \alpha$, we get:
$$\mathbb{E}_{{\color{red} \bold{\Phi}} \mid \bold{x}_{ft}}\left[\mathbb{E}\left[\mathbf{d}_{nft}\mid\mathbf{\Theta}, {\color{red} \bold{\Phi}}, \bold{x}_{ft}, \alpha\right]\right]
= \mathbb{E}_{{\color{red} \bold{\Phi}} \mid \bold{x}_{ft}}\left[{\color{red} \phi_{nt}}\bold{Y}_{nft}\left(\sum_{n^{\prime}=1}^{N}{\color{red} \phi_{n^{\prime}t}}\bold{Y}_{n^{\prime}ft}\right)^{-1}
\right]\bold{d}_{ft}$$
The linear stability of laws is preserved but not linearity of parameters for vector case
use the conditional Gaussianity as an alternative representation
III: Parameter Estimation
Lower-bound
We develop a Majorization-Minimization algorithm for the parameter estimation
Consider the LL: $\log p(\bold{X} |\bold{\Theta},\alpha) = \log \int p(\bold{X} |\bold{\Theta}, \bold\Phi) p(\bold\Phi) d\bold\Phi$
We discretize and minimize the LL by the following lower-bound:
$ \log \, p(\bold{X}|\bold{\Theta})\ge -\frac{1}{P}\sum_{f,t,m=1}^{F,T,M}\sum_{p=1}^{P}\left(\frac{\tilde{d}_{ftm}}{\tilde{y}_{ftmp}} +\log\tilde{y}_{ftmp}\right)
+ T\sum_{f=1}^{F}\log\left|\bold{Q}_{f}\bold{Q}_{f}^{\mathsf{H}}\right|-\mathrm{KL}[q(\phi_{nt})\|p(\phi_{nt})]$
$\text{KL}$ denotes the Kullback-Leibler divergence
$\tilde{d}_{ftm} = \left|\bold{q}_{fm}^{\mathsf{H}}\bold{d}_{ft}\right|, \tilde{y}_{ftmp} \!=\! \sum_{n,k=1}^{N,K} \tilde{\phi}_{ntp} w_{nkf} h_{nkt} \tilde{g}_{nm}$
$\forall p, \tilde{\phi}_{ntp}$ are realizations of $p\left(\phi_{nt} \mid \bold{X}, \bold{\Theta}, \alpha\right)$
$q\left(\theta\right)$ satisfies the equality with the LL when $ q(\theta) = p\left(\phi \mid \bold{X}, \bold{\Theta}\right)$
Variational inference procedure
E-Step: computation of $\bold\Phi$
$p\left(\phi_{nt} \mid \bold{X}, \bold{\Theta}, \alpha\right)$ is intractable
Metropolis-Hastings algorithm is used to get samples $ \tilde{\phi}_{ntp}$
Metropolis-Hastings Procedure
Draw samples from
$\phi_{nt,f}^{\text{new}}\sim\mathcal{PS}^\alpha\left(2\cos\left(\frac{\pi\alpha}{4}\right)^{2/\alpha}\right)$
Sample $\nu\sim\mathcal{U}\left(\left[0,1\right]\right)$ from the uniform distribution
Compute the acceptance probability for all $f$:
$\text{acc}\!\left(\!\phi_{nt,f}^{\text{old}}\!\rightarrow\!\phi_{nt,f}^{\text{new}}\!\right) = \min\!\left(\!1,\frac{u_{nft}\left(\phi_{nt,f}^{\text{\text{new}}}\right)}{u_{nft}\left(\phi_{nt,f}^{\text{\text{old}}}\right)}\!\right)$
$u_{nft}$: Gaussian distribution PDF
Acceptance test:
if $\nu\!<\!\text{acc}\!\left(\phi_{nt,f}^{\text{old}}\!\rightarrow\!\phi_{nt,f}^{\text{new}}\right)$, $\phi_{nt,f}\!=\!\phi_{nt,f}^{\text{new}}$ (accept)
otherwise, $\phi_{nt,f}\!=\!\phi_{nt,f}^{\text{old}}$ (reject)
Average by computing $\tilde{\phi}_{ntp} \triangleq \frac{1}{F}\sum_{f=1}^{F}\phi_{nt,f}$
M-Step: estimation of $\Theta$
Multiplicative update rules similar to AR-FastMNMF:
$$\tilde{d}_{ftm} = \left|\bold{q}_{fm}^{\mathrm{H}}\bold{d}_{ft}\right|^{2};
\tilde{y}_{ftmp} = \sum_{n,p=1}^{N,P}{\color{red}\tilde{\phi}_{ntp}}\lambda_{nft}\tilde{g}_{nm};
\bold{V}_{fm} = \frac{1}{TP}\sum_{t,p=1}^{T, P}\bold{d}_{ft}\bold{d}_{ft}^{\mathrm{H}} \tilde{y}_{ftmp}^{-1}
$$
$w_{nfk}\leftarrow
w_{nfk}\sqrt{\frac{\sum_{t, m, p=1}^{T, M, P}{\color{red}\tilde{\phi}_{ntp}} h_{nkt}
\tilde{g}_{nm} \tilde{d}_{ftm}\tilde{y}_{ftmp}^{-2}}
{\sum_{t, m, p=1}^{T, M, P} {\color{red}\tilde{\phi}_{ntp}}h_{nkt} \tilde{g}_{nm} \tilde{y}_{ftmp}^{-1}}};$
$h_{nkt}\leftarrow h_{nkt}\sqrt{\frac{\sum_{f, m, p=1}^{F, M, P} {\color{red}\tilde{\phi}_{ntp}}w_{nfk}
\tilde{g}_{nm} \tilde{d}_{ftm}\tilde{y}_{ftmp}^{-2}}
{\sum_{f, m, p=1}^{F, M, P} {\color{red}\tilde{\phi}_{ntp}}w_{nfk} \tilde{g}_{nm} \tilde{y}_{ftmp}^{-1}}};
$
$\tilde{g}_{nm}\leftarrow \tilde{g}_{nm}\sqrt{\frac{\sum_{f, t, m, p=1}^{F, T, M, P}{\color{red}\tilde{\phi}_{ntp}} \lambda_{nft}
\tilde{d}_{ftm}\tilde{y}_{ftmp}^{-2}}
{\sum_{f, t, m, p=1}^{F, T, M, P}{\color{red}\tilde{\phi}_{ntp}} \lambda_{nft}\tilde{y}_{ftmp}^{-1}}}$
$\bold{q}_{fm}\leftarrow(\bold{Q}_f \bold{V}_{fm})^{-1}\bold{e}_m;\quad
\bold{q}_{fm}\leftarrow(\bold{q}_{fm}^{\mathrm{H}}\bold{V}_{fm}\bold{q}_{fm})^{-\frac{1}{2}}\bold{q}_{fm}$
Same update of $\bold{B} \triangleq \{\bold{B}_{fl}\}_{f,l=1}^{F,L}$ where
$\tilde{y}_{ftm}$ are replaced by $\frac{1}{P}\sum_{p=1}^{P}\tilde{y}_{ftmp}$
Estimation of $\alpha$
$\alpha$-stable distributions also admits a generalized central limit theorem
A robust estimation of $\alpha$ can be given for i.i.d. realizations
Estimation method
Let assume that $\bold{X}$ contains $T$ time frame
Define $B \!\triangleq\! \left\{\bold{x}_{ft}\right\}_{f,t=1}^{F,T^\prime} \triangleq \left\{\bold{x}_{b}\right\}_{b=1}^{F,T^\prime}$ a non-overlapping set of $T^{\prime} < T$ frame
The shape matrices $\bold{R}_{ft}$ are assumed to be i.i.d
Random split of B into $B_{2}$ minibatches of $B_{1}$ samples with $|B| = B_{1}B_{2}$
$\hat{\alpha}_{B}$ is given by ($\bold\xi_{b^\prime} \!\triangleq\! \sum_{b^{\prime\prime}=1}^{B_{1}} \bold{x}_{b^{\prime\prime} + (b^{\prime}-1)B_{1}}$):
$ \frac{1}{\widehat{\alpha}_{B}} \!\triangleq\! \frac{1}{\log B_1} \!\left(\! \frac{1}{B_2}\!\sum_{b^\prime=1}^{B_2}\log\left\lVert\bold{\xi}_{b^\prime}\right\rVert \!-\! \frac{1}{\left|B\right|}\!\sum_{b=1}^{\left|B\right|}\log\left\lVert\bold{x}_{b}\right\rVert \!\right),
$
IV - Speech Enhancement and Automatic Speech Transcription Evaluation
Settings
Dataset description
REVERB CHALLENGE dataset sampled at $16~$kHz recorded with $8$ microphones
RT$_{60}$ are either 0.25, 0.5 or 0.7s
3 signal to noise ratio (SNR) level: $0, 5$ dB
$2$ distances: "near" (50cm between mic and speaker) & "far" ( $\simeq$ 2m)
$M \in \left\{2, 8\right\}$ are considered with $N=M$ (to include AR-ILRMA)
enhanced speech source selected as the one having the hightest average energy
$20$ utterances for all cases
Scores
Signal to Distorsion Ratio (SDR), Perceptual Evaluation of Speech Quality (PESQ)(higher is better) and word error rate ratio (lower is better)
Transformer-based ASR from Speechbrain is used
Baseline Methods and Configuration
Methods
$\alpha$-AR-FastMNMF : Proposed method for $\alpha < 2$ and AR-FastMNMF when $\alpha=2$
$\alpha$-AR-ILRMA : Proposed ILRMA version for $\alpha < 2$ and AR-ILRMA when $\alpha=2$
Settings
$200$ iterations for the EM algorithm with $50$ first iterations using AR-FastMNMF
NMF coefficients are randomly initialized
Demixing matrix in AR-ILRMA and $\bold{Q}_{f}$ are initialized as identity matrix $\forall f$
Tap length $L\in\{0,4\}$ and delay $\Delta=3$
$T^{\prime} = 100$ for $\alpha$ estimation
$P=10$ MH sampling for $40$ iterations with a burning period of $30$
Rank 1 constraint SCM
This init was chosen because it is the one that produces the best results for FastMNMF
SDR/PESQ/WER performances
$L=0$, ILRMA sometimes achieves the best WER score.
Combination of AR + $\alpha$ + FastMNMF achieves the best scores in general
Some instabilities when $\alpha$ becomes small because of a sampling containing a wide range value (more MH could solve this issue)
Impact of $\alpha$ on spectrogram estimation
if $\alpha$ is too small, some phoneme harmonics may vanished
$\alpha_{opt}$ depicts a spectrogram less noisy than Gaussian in silence
Speech enhancement demo
SNR=$0$dB, far distance ($\simeq 2m$), $RT_{60}=0.5s$
           Noisy speech
        Clean Speech
$\alpha$-AR-ILRMA
               $\alpha=2$
              $0<\alpha< 2$
$\alpha$-AR-FastMNMF
               $\alpha=2$
              $0<\alpha< 2$
As a reminder, ILRMA is only determined
more background noise
close to no noise for the bus alpha. just remains a bit of noise in the high frequency
V - Conclusion and Future Works
Conclusion & Future Works
Conclusion
Extension from Gaussian AR-FastMNMF to AR-$\alpha$-FastMNMF
Interprtability of impulse variable for dereverberation
Compare to previous studies, automatic optimization of $\alpha$
Future works
Current parameter model
Replace $\alpha$ by $\alpha_{t}$ to model the dynamic range along time
Deep Neural Network extensions
Replace the NMF speech model by a heavy-tailed deep speech prior $\color{black}\text{[Fon. 19]}$
Use a normalizing flow applied on FastMNMF decomposition $\color{black}\text{[Nug. 20]}$
A.A. Nugraha, K. Sekiguchi, M. Fontaine, Y. Bando, K. Yoshii (SPL, 2020) Flow-Based Independent Vector Analysis for Blind Source Separation.
M. Fontaine, A.A. Nugraha, R. Badeau, K. Yoshii and A. Liutkus (EUSIPCO, 2019). Cauchy Multichannel Speech Enhancement with a Deep Speech Prior.
multi alpha: different dynamic for each source
using the so-called covariation, a generalization of the covariance to develop a update parameter directly in the alpha stable representation
in Gaussian case, a local gaussian model (TF bin and independent where the variance represents the PSD) is derived
from the assumption in the time domain of a WSS-GP (stationnary). The extension in alpha-stable is alpha-harmonizable process in the time domain
Thank you for your attention ! Questions ?


#include <MOFSVar.h>
Classes | |
| struct | VarValues |
| struct | XArray |
Public Member Functions | |
| float | begin_interval (char set) |
| void | calc () |
Function: calc More... | |
| int | calc_values (float x) |
Function: calc_value More... | |
| float | center (char set) |
Function: center More... | |
| char | centerSet () |
| int | correctValue (char set) |
Function: correctValue More... | |
| float | crispCentral () |
| float | crispIndex () |
| float | end_interval (char set) |
| float | maxCentral () |
| float | maxX () |
| float | minCentral () |
| float | minX () |
| MOFSVar (char *_name, float _max_x, float _min_x, int crisp) | |
| MOFSVar (char *_name, float _max_x, float _min_x, int crisp, float _max_central, float _min_central, int _crisp_central) | |
| MOFSVar (FILE *file_in, int option) | |
| void | Outputs (float y_array[]) |
| void | SalvarIncidencia (FILE *file_out) |
| char | setIndex () |
| void | SetsArray (char sets_array[]) |
| int | values_lenght () |
| void | varName (char n[]) |
| void | VName (char name_[]) |
| float | xValue () |
| float | yValue (char set) |
| ~MOFSVar () | |
Function: ~MOFSVar More... | |
Private Attributes | |
| char | _name [30] |
| int | array_size |
| float | crisp_central |
| int | crisp_central_value |
| float | crisp_index |
| int | crisp_value |
| MOFSFuncTri * | function_type |
| float | input |
| float | max_central |
| int | max_str |
| float | max_x |
| float | max_y |
| float | min_central |
| float | min_x |
| float | min_y |
| char | name [30] |
| int | overlap_index |
| char | set_index |
| float | set_lenght |
| VarValues * | values |
| XArray * | x_values_array |
| MOFSVar::MOFSVar | ( | char * | _name, |
| float | _max_x, | ||
| float | _min_x, | ||
| int | crisp | ||
| ) |
Function: MOFSVar
Purpose: 1st constructor. Arguments: _name --> the name of the var _max_x --> the maximum value for sets in the x axis _min --> the minimum value for sets in the x axis crisp --> number of sets Returns: nothing
Definition at line 60 of file MOFSVar.cpp.
| MOFSVar::MOFSVar | ( | char * | _name, |
| float | _max_x, | ||
| float | _min_x, | ||
| int | crisp, | ||
| float | _max_central, | ||
| float | _min_central, | ||
| int | _crisp_central | ||
| ) |
Function: MOFSVar
Purpose: 2nd constructor. Arguments: _name --> the name of the var _max_x --> the maximum value for sets in the x axis _min --> the minimum value for sets in the x axis crisp --> number of sets _max_central --> the maximum value for sets of central zone with a different number of sets _min_central --> the minimum value for sets of central zone with a different number of sets _crisp_central --> number of sets of the central zone Returns: nothing Comments: In this constructor is possible to define a zone with specific number of sets
Definition at line 115 of file MOFSVar.cpp.
| MOFSVar::MOFSVar | ( | FILE * | file_in, |
| int | option | ||
| ) |
Function: MOFSVar
Purpose: 3nd constructor. Arguments: file_in --> The file with the information stored option --> 0: The sets are generated autonomously using the information stored (equal to the 2nd constructor arguments) 1: The sets are read from the file, so you can specify the exactly position and size of each set Returns: nothing Comments: In this constructor is possible read all the information from a file (recommended option)
Definition at line 160 of file MOFSVar.cpp.
| MOFSVar::~MOFSVar | ( | ) |
Function: ~MOFSVar
Purpose: desconstructor. Arguments: none Returns: nothing Comments: nothing to do right now
Definition at line 228 of file MOFSVar.cpp.
| float MOFSVar::begin_interval | ( | char | set | ) |
Definition at line 337 of file MOFSVar.cpp.
| void MOFSVar::calc | ( | ) |
Function: calc
Purpose: Calc all the values of the x_values_array Arguments: none Returns: nothing Comments: Calc all the x's values and the set values
Definition at line 248 of file MOFSVar.cpp.
| int MOFSVar::calc_values | ( | float | x | ) |
Function: calc_value
Purpose: Calc the sets and y values for the input x Arguments: none Returns: nothing Comments: Calc all the x's values and the set values
Definition at line 360 of file MOFSVar.cpp.
| float MOFSVar::center | ( | char | set | ) |
Function: center
Purpose: Return the center value of the set Arguments: char set --> the set which center is wanted to konw Returns: the center of the set
Definition at line 318 of file MOFSVar.cpp.
| char MOFSVar::centerSet | ( | ) |
Definition at line 497 of file MOFSVar.cpp.
| int MOFSVar::correctValue | ( | char | set | ) |
Function: correctValue
Purpose: check if a set is set value is valid for the last entry Arguments: set --> The set we want to know Returns: -1 --> If there is no entry 0 --> If the set is not valid 1 --> valid set
Definition at line 448 of file MOFSVar.cpp.
| float MOFSVar::crispCentral | ( | ) |
Definition at line 593 of file MOFSVar.cpp.
| float MOFSVar::crispIndex | ( | ) |
Definition at line 580 of file MOFSVar.cpp.
| float MOFSVar::end_interval | ( | char | set | ) |
Definition at line 328 of file MOFSVar.cpp.
| float MOFSVar::maxCentral | ( | ) |
Definition at line 585 of file MOFSVar.cpp.
| float MOFSVar::maxX | ( | ) |
Definition at line 572 of file MOFSVar.cpp.
| float MOFSVar::minCentral | ( | ) |
Definition at line 589 of file MOFSVar.cpp.
| float MOFSVar::minX | ( | ) |
Definition at line 576 of file MOFSVar.cpp.
| void MOFSVar::Outputs | ( | float | y_array[] | ) |
Definition at line 528 of file MOFSVar.cpp.
| void MOFSVar::SalvarIncidencia | ( | FILE * | file_out | ) |
Definition at line 540 of file MOFSVar.cpp.
| char MOFSVar::setIndex | ( | ) |
Definition at line 553 of file MOFSVar.cpp.
| void MOFSVar::SetsArray | ( | char | sets_array[] | ) |
Definition at line 513 of file MOFSVar.cpp.
| int MOFSVar::values_lenght | ( | ) |
Definition at line 487 of file MOFSVar.cpp.
| void MOFSVar::varName | ( | char | n[] | ) |
Definition at line 560 of file MOFSVar.cpp.
| void MOFSVar::VName | ( | char | name_[] | ) |
Definition at line 569 of file MOFSVar.cpp.
| float MOFSVar::xValue | ( | ) |
Definition at line 548 of file MOFSVar.cpp.
| float MOFSVar::yValue | ( | char | set | ) |
Definition at line 464 of file MOFSVar.cpp.
|
private |


Thinking styles
How to adapt your thinking style and evolutionize.
May 18, 2015
Topics:
- Types of thinking styles
- Andre's way to adapt!
Thinking styles
There are various thinking styles and lets list them really quick so you get glimpse at them: -Concrete Sequential: tend to be based in reality. They process information in an ordered, sequential linear way. -Concrete Random: EXPERIMENTERS. -Abstract Random: organize information thru reflection and thrive in unstructured, people-oriented environments. -Abstract Sequential: love the world of theory and abstract.
When I took the test it said that I am an Abstract Random thinker. So basically what I understand and feel when I think is that ideas just pop right out of my head and I try tu pt the pieces together and start giving shots until the right anwser comes to my mind. So basically I consider myself a little distracted sometimes but when I start solving problems ideas just pop out of my head and I begin with trial and error. Also I learn a lot with experience so if I want to learn something jsut practice it many times and with different cases or in different ways. Literally practice keeps my mind sharp because my memory is really bad.
So far, DBC has been awesome and what I learn I always keep track in my HUGE notebook. See that the thing I need to experiment and practice so that is why i write everything down right AFTER I understand it and if not I write it down for asking later. The biggest struggle right now is putting in practic all that we have learned in these three weeks but so far it is looking good.
Fixed vs Growth Mindset
So this is a very important topic apparently in DBC... and I cant agree more. This is what determines your success in life how far do you want to go. Do you know how far you can go? NO the anwser is NO and that is because you can estimate your capabilities but not as a fact so you can always push yourself to the limit and yeah... you will never know how good you can be.
A fixed mindset is someone who puts limits to himself and says I know I can go up to here and that is it I am not learning more than that. Instead Growth Mindset is someone who is abosorbing knowledge everytime and learning from their wrongs so next time they cna be better and this way improve constantly and theese type of person tends to thrive a lot more than someone who basically limits him or herself. So get out of your comfort zone and you will never know how far you can go, so just keep on going!
DBC Alumni Andres Castillo Ormaechea @devgmtech


CS7NS6 Distributed Systems 2019-20 Exercise 2
Implementing a Replicated Service Using Process Groups
Final Report by Chao Chen - 19310133
This project aim to implement a distributed key-value store based on the RAFT algorithm. The implementation is done with Java. The exact implementation, functionalities and architecture of this service will be discussed further throughout this document.
The work is based on RAFT protocol. A introduction about the RAFT algorithm can be found here: https://raft.github.io/
The original paper is here with the bibliography.
Ongaro, Diego, and John Ousterhout. "In search of an understandable consensus algorithm." 2014 {USENIX} Annual Technical Conference ({USENIX}{ATC} 14). 2014.
@misc{ongaro2013search,
title={In search of an understandable consensus algorithm (extended version)},
author={Ongaro, Diego and Ousterhout, John},
year={2013},
publisher={Tech Report. May, 2014. http://ramcloud.stanford.edu/Raft.pdf},
url={http://ramcloud.stanford.edu/Raft.pdf}
}
We are going to implement a distributed key-value store. One famous implementation is 'etcd' (https://github.com/etcd-io/etcd). The service, 'etcd', is a distributed key-value store for critical data, which are often the configurations for the cloud.
The read / write metrics are not the key concern in our requirements, as for configurations they are often accessed when one service in the cloud is initializing. What we do concern is the ability to provide non-stop services. It requires the system to have fault tolerance and the ability to resize dynamically when on line. As for storage systems, data consistency is also important.
For service management:
For service operation:
For replica failures:
For partitioning:
For client:
Graph drawing using draw.io
(https://app.diagrams.net/).
A big reason for using Java is that it provides a synchronized
keyword to delare methods, preventing them to be invoked at the same time by multiple threads, or we can say the methods can be invoked when the thread get the lock.
Brief information about all the modules:
There are 3rd party libraries involved in our project.
The communication is based on RPC calls. We use sofa-bolt
from Alipay (https://github.com/sofastack/sofa-bolt). The connection is based on TCP with a compact data package defined by the library.
The storage is implemented with a built-in database library called RocksDB
(https://github.com/facebook/rocksdb) from Facebook. The database is based on file system so we can persist the data and logs on the disk when nodes fail. Another feature is that the RocksDB get operation on single key is thread safe (https://github.com/facebook/rocksdb/wiki/Basic-Operations#concurrency) while for set operation we need to use external locks.
The node state decides the behavior across the modules. 3 states are divided for implementing the RAFT protocol.
The every node starts as a Follower. And in our implementation, the first node in the group, it will go through start -> Follower -> Timeout, Start Election -> Candidate -> Recuive Majority Votes -> Leader
. Then the rest of nodes (real Followers) can join the group with the comunication with Leader (we will explain the resizing later).
Leader will handle all the Get/Set/Del(ete) operations from client, request to the Followers will be redirected to the Leader. For Set/Del operations, Leader will need to commit the changes to at least half of the nodes in the group. A log entry conatins a term number, a incremental index number and the operations performed on the database (State Machine), and client will receive the success result when harf of the nodes in the group commited the change.
Some terms explained in Raft protocol.
Terms are marked with incremental integers. Each term begins with an election, in which one or more candidates attempt to become leader.
Index is a incremental integer to mark the order of a log entry.
Log entries are records with a term, an index and a command (Set/Del) from the client. Data modification generates log entries, and they are used to do data replication and recovery.
The node maintains the term
number of itself and a commitIndex
. Term number is incremental for deciding the which node is up to date to become the Leader. commitIndex
is the commited index number of log entries for the node itself.
There are two scheduled tasks running inside the node logic.
The heartbeat task only start from a Leader. In normal situation, when Follower receives the heartbeat from Leader, the election timeout will be reset (no need for election when Leader is alive). Term number and commitIndex from the leader are synced to the Followers.
The election task start when the Follower is not receiving the heartbeat from Leader. Followers and Candidates will vote for the requester if themselves' term number is less equal than the requester.
When receiving no heartbeat from Leader, the follower will start a ElectionTask as a Candidate. Itself's currentTerm
will increase, then vote for itself, and send vote reuests with: currentTerm
, lastLogIndex
, lastLogTerm
The request is handled by Consensus Module.
Once gathered votes for majority >= n/2
, it will become Leader.
The leader keeps track of all the commitIndex numbers from Followers.
Beside timeouts for election and heartbeat, Node Logic also provide handlers from Communication module.
Almost all the handler implementations with writes on the properties of the node are synchronized, which mean that a type of handler can be only invoked for 1 at a time.
The Consensus module, in charge of two functionality:
appendEntries
: The method, evaluate the log entries sent from LEADER and append them to log module.
To note that appendEntries
will also handle heartbeat (heartbeat comes without log entries). The timeout for election task will be reset. If the requester’s term is newer, the handler node will become Follower. If handler node’s term is newer, the requester will become Follower.
Term numbers and index of the log entry are evaluated and then applied. If the previous index from the requester is not the same as the handler’s last index, the method will fail, and ask the requester to retry with a older (index - 1) log index to do replication.
requestVote
: The method handles requests for vote from other Candidate.
If there is no record for voting, the Candidate’s term and last index number will be compared.
If the handler node’s term is bigger/newer than the Candidate, fail the request and the Candidate’s Term will be updated.
If the handler node’s last log index or last log term is bigger/newer than Candidate’s, the request will also fail.
The State Machine and Log Module provide similar functionalities towards storing and retrieving data, since both of them is implemented by wrapping RocksDB. Data and log entries are persisted on disk, so we can lower the risk of losing data when there is a node failure.
But when before adding a node or after deleting the node, we need to make sure whether the data and log entries remain on the disk are needed. The data will be recovered and may produce weird behavior on the index and data consistency.
A basic implementation of group resizing is done by adding and syncing the list of peers transmitted along with the heartbeat from leader:
This rough implementation breaks the partition behavior in RAFT protocol. The Raft protocol only ensures the partition of majority (>n/2) work fine with the Follower-Candidate-Leader state change. We resized the members so the total number n
is dynamic and breaks this behavior in RAFT.
The client is implemented by continuous reading the input of the console. Multiple lines can be accepted and each line is processed by a thread in the thread pool (max=20, hardcoded), so we can copy prepared scripts into the console to do requests in batch. Commands of get/set/delete can be parsed and sent by specifying the target node with host and port. For test convinience, a setFail command is added to the requirement/specification for switch the simulation of node (network) failure from one node towards one other node in the group, hense we can manually build partitions by this command.
Also for test convinience we added a webpage GUI to perform request from client.
We implemented a rough distributed key-value store based on RAFT protocol. The system is built with considerations about concurrency, with locks and synchronized queues as the main concurrency model. Dynamic group building and resizing is managed. Data and log entries are persisted on disk. A leader election mechanism is implemented, and the leader take in charge of the decisions like commitment. Failures are tolerated and multiple partitions are supported.
There are still some future work remains. The dynamic resizing feature is rough and dangerous. In the RAFT paper (section 6), there is a solution with 2 phase commiting, in which the old and new verions of configurations are managed. Merging the partitions are still a problem. For log entries we did not do compaction. Much effort is needed for these aims.
Please use at least jdk 1.8
to compile and run the program, with the MAVEN (mvn
) toolchain installed.
To compile, run the compiling script compile.sh
in the root of the project folder, a jar file KVNode.jar
will appear at the root.
$ sh compile.sh
The builds of the code are divided into:
-c
flag-c
flag and with valid configsUse -h
or --help
command to access the help.
$ java -jar KVNode.jar -h
usage: KVNode.jar
-c,--client Initiate as a client.
-h,--help Help
-o,--host <arg> The host of server
-p,--port <arg> The port of server listening to
-t,--target-host <arg> The host of target server to join, omit this or
target-port to establish a new group (as the first and
the leader)
-y,--target-port <arg> The port of target server to join, omit this or
target-host to establish a new group (as the first and
the leader)
To run client, use:
$ KVNODENAME=CLIENT java -jar KVNode.jar -c
To run a server node, use: (where 4111
is the port listening, while 4110
is the target/leader port)
$ KVNODENAME=4111 java -jar KVNode.jar -p 4111 -y 4110
The host/IP localhost
is omitted.
The ENV variable KVNODENAME
is for logs (both program log and log entries) and data persistence on the file system:
KVNODENAME.log
: program log / outputKVNODENAME_state/
: data storageKVNODENAME_log/
: storage for log entriesDo notice to start nodes ONE BEFORE OTHERS with LEADER - FOLLOWER order to establish the cluster.
For example:
4111
first (without the flag -y
): KVNODENAME=4111 java -jar KVNode.jar -p 4111
4111
is ready as a LEADER, start another server (4112
) using: KVNODENAME=4112 java -jar KVNode.jar -p 4112 -y 4111
Also be care about the RocksDB files remain in the file system. Since data and logs are persisted, it might bring impact to the experiment. Clean them before experiment.
Program in client mode will read the lines input and parse the command.
ip:port
is to determine which SERVER/NODE is going to communicate:
ip:port get KEY
KEY
ip:port set KEY VALUE
VALUE
to the KEY
ip:port del KEY
KEY
(actually implemented with ip:port set KEY null
)ip:port setFail ip2:port2
ip2:port2
For example, getting a value of key foo
from a node(localhost:4111
):
localhost:4111 get foo
For easy recognition the log prefixes with dates and class names (12:07:42.308 [pool-1-thread-2] INFO life.tannineo.cs7ns6.App
) are omitted.
Start a 3-instance cluster with leader first decided, we can see without specifying the complete list of nodes/peers the cluster still can be built. And nodes in service can be terminated as you want. A simplified (also dangerous) dynamic management based on detecting connection failures (e.g. heartbeat) is implemented.
$ KVNODENAME=4111 java -jar KVNode.jar -p 4111 # as leader
$ KVNODENAME=4112 java -jar KVNode.jar -p 4112 -y 4111
$ KVNODENAME=4113 java -jar KVNode.jar -p 4113 -y 4111
Start a 3-instance cluster with leader first decided, use the client to execute the command one by one:
localhost:4112 get foo
localhost:4111 set foo bar
localhost:4113 get foo
We can see that we can get / set on different nodes.
...
- You just input:
localhost:4112 get foo
- request content : foo, url : localhost:4112, put response : ClientKVAck(result=null)
- get foo=null
localhost:4111 set foo bar
- You just input:
localhost:4111 set foo bar
- request content : foo=bar, url : localhost:4111, put response : ClientKVAck(result=ok)
- set foo ok
localhost:4113 get foo
- You just input:
localhost:4113 get foo
- request content : foo, url : localhost:4113, put response : ClientKVAck(result=Command(opt=null, key=foo, value=bar))
- get foo=bar
Close localhost:4111
, wait for about 10 sec (election task timeout), the rest 2 nodes will go for election. Then update data on localhost:4112
:
localhost:4112 set foo barbar
When successed, start localhost:4111
to join the cluster as a follower: KVNODENAME=4111 java -jar KVNode.jar -p 4111 -y ?WHOSTHELEADER?
, we can see the index persist and then updated by 1 when join the cluster (because we did one change). Then execute query on client:
localhost:4111 get foo
We can get barbar
, the new data.
...
localhost:4111 get foo
- You just input:
localhost:4111 get foo
- request content : foo, url : localhost:4111, put response : ClientKVAck(result=Command(opt=null, key=foo, value=barbar))
- get foo=barbar
For partition, we test it on a 5-instance cluster. Remove all the disk files generated for clean experiment, and start 5 nodes.
$ KVNODENAME=4111 java -jar KVNode.jar -p 4111 # as leader
$ KVNODENAME=4112 java -jar KVNode.jar -p 4112 -y 4111
$ KVNODENAME=4113 java -jar KVNode.jar -p 4113 -y 4111
$ KVNODENAME=4114 java -jar KVNode.jar -p 4114 -y 4111
$ KVNODENAME=4115 java -jar KVNode.jar -p 4115 -y 4111
In the client, we execute the commands in batch (the client command executor is also implemented concurrently, and can parse multiple lines).
localhost:4111 setFail localhost:4113
localhost:4111 setFail localhost:4114
localhost:4111 setFail localhost:4115
localhost:4112 setFail localhost:4113
localhost:4112 setFail localhost:4114
localhost:4112 setFail localhost:4115
localhost:4113 setFail localhost:4111
localhost:4113 setFail localhost:4112
localhost:4113 setFail localhost:4115
localhost:4114 setFail localhost:4111
localhost:4114 setFail localhost:4112
localhost:4114 setFail localhost:4115
localhost:4115 setFail localhost:4111
localhost:4115 setFail localhost:4112
localhost:4115 setFail localhost:4113
localhost:4115 setFail localhost:4114
The node are divided into 4111 & 4112 | 4113 & 4114 | 4115
. n = 3
...
- setFail localhost:4111 ok -> true
- setFail localhost:4113 ok -> true
- setFail localhost:4112 ok -> true
- setFail localhost:4113 ok -> true
- setFail localhost:4115 ok -> true
- setFail localhost:4114 ok -> true
- setFail localhost:4114 ok -> true
- setFail localhost:4115 ok -> true
- setFail localhost:4115 ok -> true
- setFail localhost:4115 ok -> true
- setFail localhost:4111 ok -> true
- setFail localhost:4112 ok -> true
- setFail localhost:4111 ok -> true
- setFail localhost:4112 ok -> true
- setFail localhost:4113 ok -> true
- setFail localhost:4114 ok -> true
true
means start blocking. Execute the same command again to turn it into false
(like a switch).
Wait for election happens among the nodes.
Partition merging features are not implemented.


COVID- neurological patients stroke SARS-CoV- disease patient ahead April care coronavirus â€". associated manifestations respiratory case risk symptoms {border-style infection acute print px;} #ffffff;} Case Epub nervous pandemic However, accessed healthcare report severe COVID-. al. complications syndrome collab: CSF cases patients, potential SARS-CoV-. cerebrovascular clinical direct diseases impact management study Engl Neurological Stroke left online performed print, series treatment use March Med. Neurol RT-PCR brain cerebral characteristics emergency facial factors infected neurology present px; reported {font-family Apr COVID-, Neurology. central currently data disorders font-size high human intracerebral ischemic medical multiple neurologists pandemic. positive showed time ./WNL.. CNS SARS-CoV addition, common developed double; hospitalized infection. intensive manifestations, middle patients. peripheral possible presented presenting psychological syndrome. viral virus workers (.%) (COVID-) Adv Med; Patients [published artery blood border-color border-width confirmed critical day early especially health higher hospitalization infections majority mechanisms need negative nerve personnel pneumonia potentially prolonged recent related significant solid; studies swab unit worldwide {font-style {font-weight (%) (SPC). (Table ./NEJMc. EMC, Med Medicines.org.uk. Parkinsonâ€™s SARS [Published admitted analysis association bold care, causal cause chest cohort concomitant contribute courier; death disease, diseases, east em; encephalitis encephalitis. epidemics evaluation examination fact, failure general haemorrhage headache headache, history increase indicative indirect infectious information involved involvement known mild nasopharyngeal neurodegenerative novel outcomes previous product pt;} recovery relevant reports review right sclerosis stroke, subsequent symptoms, symptoms. temporal tested understanding used visits âˆ’


| Home | Libraries | People | FAQ | More |
WritablePropertyMap
A WritablePropertyMap is used for writing a value at a position, as represented by a cursor.
elements
BinaryFunc
Must be a model of BinaryFunction.
cursor
Iter
Must be a model of ReadableIterator and IncrementableIterator.
value_type
Value
Must be a model of ReadableIterator and IncrementableIterator.
key_type
ReadableIterator<Iter>::reference
The key type of the cursor
| Copyright © 2006 Eric Niebler |


Time Series Lab
Notebooks
Add notebook
#
Name
1
Prediction
2
Anomalies
3
Topology finder
4
I&I analysis


Calum Best: I’m super picky about dating now
Calum Best says he no longer goes on so many dates.
The son of the late football star George Best admitted he saw a lot of women when he was in his 20s.
“In my younger years, I dated all the time. Nowadays, I’m single still but I don’t date that often, I’m super picky … because I don’t want to waste their time or my time,” he told podcast The Truth Flirts.
“In my early 20s I was out every night going on dates.”
Best, who appeared on Celebrity Love Island, added: “I’m 38 now and I’m single and in my head I’m like, ‘The one is there somewhere’.
“Sometimes I mess up and put myself a time limit and I’m like ‘Ooh, if it’s not happening by the time I’m 40 I’m gonna think of all these alternatives’.”
Mel B, who hosts the podcast, said of dating multiple people: “I think in an ideal world, you’d have this person there for your emotional support, you’d have that person there for your intimacy.
“I would like to think that there’s one person that could encompass everything but three marriages later, three kids later, I’m thinking is there ever that one person that just has everything?”
Mel B also spoke to Love Island’s Kaz Crossley.
Podcast The Truth Flirts, launched with dating app Badoo, is at http://bit.ly/TheTruthFlirts.


Examples¶
Basic Graph¶
"""Basic VTGraph usage example."""
from vt_graph_api import VTGraph
API_KEY = "" # Insert your VT API here.
# Creates the graph.
graph = VTGraph(API_KEY, private=False, name="First Graph")
# Adds the node.
graph.add_node(
"ed01ebfbc9eb5bbea545af4d01bf5f1071661840480439c6e5babe8e080e41aa",
"file", label="Investigation node")
# Expands the graph 1 level.
graph.expand_n_level(level=1, max_nodes_per_relationship=5, max_nodes=100)
# Saves the graph.
graph.save_graph()
# Get the graph id.
print("Graph Id: %s" % graph.graph_id)
# Visualizing the Graph.
print(graph.get_ui_link()) # Open the url in the browser
Advanced Graph¶
"""Advanced VTGraph usage example."""
from vt_graph_api import VTGraph
from vt_graph_api.errors import NodeNotFoundError
API_KEY = "" # Add your VT API Key here.
# Make sure you have private graph quota. Otherwise it will fail to save
graph = VTGraph(
API_KEY, verbose=False, private=True, name="First Graph",
user_editors=["jinfantes"], group_viewers=["virustotal"])
# Adding first node, WannyCry hash.
graph.add_node(
"ed01ebfbc9eb5bbea545af4d01bf5f1071661840480439c6e5babe8e080e41aa",
"file", label="Investigation node")
print("Expanding... this might take a few seconds.")
graph.expand_n_level(level=1, max_nodes_per_relationship=10, max_nodes=200)
# Adding second node, Kill Switch domain.
graph.add_node(
"www.ifferfsodp9ifjaposdfjhgosurijfaewrwergwea.com", "domain",
label="Kill Switch", fetch_information=True
)
# Expanding the communicating files of the kill switch domain.
graph.expand(
"www.ifferfsodp9ifjaposdfjhgosurijfaewrwergwea.com", "communicating_files",
max_nodes_per_relationship=20
)
# Deleting nodes.
nodes_to_delete = [
"<IP_ADDRESS>",
"<IP_ADDRESS>",
"<IP_ADDRESS>",
"<IP_ADDRESS>",
"<IP_ADDRESS>",
"<IP_ADDRESS>",
"<IP_ADDRESS>",
"<IP_ADDRESS>",
"<IP_ADDRESS>",
"fb0b6044347e972e21b6c376e37e1115dab494a2c6b9fb28b92b1e45b45d0ebc",
"428f22a9afd2797ede7c0583d34a052c32693cbb55f567a60298587b6e675c6f",
"b43b234012b8233b3df6adb7c0a3b2b13cc2354dd6de27e092873bf58af2693c",
"85ce324b8f78021ecfc9b811c748f19b82e61bb093ff64f2eab457f9ef19b186",
"3f3a9dde96ec4107f67b0559b4e95f5f1bca1ec6cb204bfe5fea0230845e8301",
"2c2d8bc91564050cf073745f1b117f4ffdd6470e87166abdfcd10ecdff040a2e",
"a93ee7ea13238bd038bcbec635f39619db566145498fe6e0ea60e6e76d614bd3",
"7a828afd2abf153d840938090d498072b7e507c7021e4cdd8c6baf727cafc545",
"a897345b68191fd36f8cefb52e6a77acb2367432abb648b9ae0a9d708406de5b",
"5c1f4f69c45cff9725d9969f9ffcf79d07bd0f624e06cfa5bcbacd2211046ed6"
]
for node in nodes_to_delete:
try:
graph.delete_node(node)
except NodeNotFoundError:
pass # Ignoring if the node does not exist in the graph.
graph.save_graph()
print("Graph ID: %s" % graph.graph_id)
Basic Graph Search¶
"""VTGraph basic search usage example."""
from vt_graph_api import VTGraph
API_KEY = "" # Insert your VT API here.
# Creates the graph.
graph = VTGraph(API_KEY, verbose=True, private=True, name="First Graph")
# Add some nodes to graph.
graph.add_node("b3b7d8a4daee86280c7e54b0ff3283afe3579480", "file", True)
graph.add_node("nsis.sf.net", "domain", True)
graph.add_links_if_match(
"b3b7d8a4daee86280c7e54b0ff3283afe3579480", "nsis.sf.net",
max_api_quotas=1000, max_depth=10)
# Try to connect node with graph.
graph.save_graph()
# Get the graph id.
print("Graph Id: %s" % graph.graph_id)
# Visualizing the Graph.
print(graph.get_ui_link()) # Open the url in the browser
Advanced Graph Search¶
"""VTGraph advanced search usage example."""
from vt_graph_api import VTGraph
API_KEY = "" # Insert your VT API here.
# Creates the graph.
graph = VTGraph(API_KEY, verbose=True, private=True, name="First Graph")
# Add some nodes to graph.
graph.add_node("b3b7d8a4daee86280c7e54b0ff3283afe3579480", "file", True)
graph.add_node("nsis.sf.net", "domain", True)
graph.add_node(
"26c808a1eb3eaa7bb29ec2ab834559f06f2636b87d5f542223426d6f238ff906", "file")
graph.add_node("www.openssl.org", "domain", True)
# Try to connect node with graph.
graph.connect_with_graph(
"b3b7d8a4daee86280c7e54b0ff3283afe3579480", max_api_quotas=1000,
max_depth=10)
graph.save_graph()
# Get the graph id.
print("Graph Id: %s" % graph.graph_id)
# Visualizing the Graph.
print(graph.get_ui_link()) # Open the url in the browser
Load Graph¶
"""VirusTotal Graph id load example."""
import vt_graph_api
API_KEY = "" # Insert your VT API here.
GRAPH_ID = "" # Insert yout graph id here.
# Retrieve the graph.
graph = vt_graph_api.VTGraph.load_graph(GRAPH_ID, API_KEY)
# Modify your graph here.
# Save it in VirusTotal.
graph.save_graph()
# Get the graph id.
print("Graph Id: %s" % graph.graph_id)
# Visualizing the Graph.
print(graph.get_ui_link()) # Open the url in the browser.


Technical blog from Craig Russell.
There are two types of links you can create in Markdown:
This post details how to use both types.
The standard syntax for creating links in markdown is providing text to display in square brackets [ ]
, followed by the URL in parentheses ( )
. For example, to link to this blog, I could do this:
[My Blog](https://craigrussell.io)
, which renders as My Blog
[ ]
and ( )
[ ]
appears before the ( )
Instead of adding display text for the link you can have it displayed as just the URL, and there are 2 ways to do this.
One way to create a link with no title is to duplicate the URL into both parts of the link: e.g.,
[https://craigrussell.io](https://craigrussell.io)
, which renders as https://craigrussell.io.
The best way to create a link which just shows the URL and no custom display text is to wrap it inside angle brackets < >
.
<https://craigrussell.io>
, which renders as https://craigrussell.io


Trickle down economics! A rising tide lifts all boats! As long as the stock market is cooking, everything else will fall in place. This is the gov't's most important job.
But how has the economy really fared?
There are a couple common misconceptions about the economy that can lead people to bad conclusions.
First, the stock market is not the economy. The major measurements of the stock market that are always bandied about are usually the Dow and NASDEQ, right?
Well, both of these are simply indices (or averaged measurements) on a group of stocks. In the case of the NASDEQ, it follows almost all of the stocks listed on the NASDEQ stock exchange.
In the case of the Dow, it measures just 30 large companies. So this raises the obvious question: Can we really say the economy is healthy just because the stock of 30 large companies is doing well?
Surely that depends on who you are. If you're a major stockholder in those companies, then you might feel like it is. But the vast majority of people don't have money invested in the stock market.
Instead, we could look at the real buying power of the average family. Since 2001, the median household income for renters rose 0.5%, while rent prices rose 13%.
At the same time, US wealth has become increasingly monopolized by a smaller and smaller percentage of people.
As
the Pew Research Center
or
the Brooking Center
will show you, the top 1% of people now have more wealth than than the entire middle class here in America, and that gap is always increasing.
Since 1995, the share of wealth held by the lower and middle income groups have decreased while the rich have gotten richer.
As you can see
here
, Trump did worse with the stock market than Obama over the same span of time in their respective presidencies.
This actually follows an
established trend reported by Forbes
, where the stock market tends to do better under Democratic presidents than under Republicans.
The climate is always changing! Volcanoes generate more carbon than humans! Environmental protections get in the way of the economy!
But what does the science really say?
The basic mechanism of global climate change is very easy to understand. Much like a greenhouse, our atmosphere lets in more solar energy than it lets out.
This is the mechanism that allows life on Earth. Without it, solar energy would just bounce away, and like other planets in our atmosphere our environment would be unlivable.
As the concentration of gases in the atmosphere grows, this basic effect becomes more pronounced. To keep with the greenhouse metaphor, adding more greenhouse gases to the atmosphere is like adding more and thicker glass to your greenhouse.
As these changes progress, the world changes in some subtle but very important ways. Places that used to receive a lot of rainfall will quickly start to see decreased annual precipitation, meaning global food stocks will be affected.
And these changes are happening already. Even here in the US,
where people so often feel that they are protected from the calamities of the wider world, the effects of climate change can already be felt in some tremendous, visceral ways.
Riots aren't the way to get what you want! Black on black crime is the real problem!
But is everyone really treated the same, regardless of skin color?
Read Ta-Nehisi Coates' Between the World and Me
Our justice and social systems are built to prop up white Americans at the expense of people of color.

But how much does healthcare really cost?
Americans already pay 4 times more per person for healthcare. And that's for a medical system that lags well behind the rest of the developed world in patient outcomes.
Our rate of infant mortality, death in childbirth, and death due to lack of care is many times higher than the rest of the developed world.
A great deal of that goes to pharmaceutical and insurance companies - third parties whose only vested interest is in keeping their stock prices high and their costs low.
Their bottom line costs you money but never lends anything to your health.
Every life is precious! Abortion is murder!
But are traditional pro-life policies really the best solution?
Abortion is one of the mostly fiercely debated topics in American politics, and for many people it is the deciding factor in the way they vote.
The traditional wisdom is that the GOP is the pro-life party, and for that reason most single-issue abortion voters are reliable Republicans.
However, study
after study show that this traditional wisdom is mistaken.
The abstinence-only sex education policies, combined with decreaed access to birth control and abortion, that are the centerpieces of Republican policy on abortion actually drive abortion rates higher.
When women are given better access to knowledge and resources, specifically safer sex practices, birth control, and knowledge of how to use it properly, abortion rates drop.
While Republican officials like to tout themselves as the pro-life party, the truth is that Democratic leaders and policies are much more effective at decreasing abortion rates.
This means that a voter truly concerned with the proliferation of abortion should step back from the party line, remember what is really at stake,
and vote for officials pushing for increased access to reproductive health care, more freedom for women to plan their families as they wish,
and increased funding to organizations like Planned Parenthood that offer a wide array of services designed to improve women and children's health.


By default Microsoft Sql Server continues batch execution if an error occurs. It skips ONLY the statement with the error.
To make sure it will rollback the transaction use SET XACT_ABORT ON before opening a transaction.

The XACT_ABORT is available at least since Sql Server 20000 so should be ok if you work with different versions of SqlServer.


Rotation 4
This page is for the fourth rotation, which was themed around the laser cutter. We could cut and engrave things with it, and were given three assignments: a clock, a regular engraving, and an egraving on a cup or mug or something round.
The first assignment, the clock, was going to be laser cut on a record so I wanted mine to have a music theme. I desgined mine with some plain music notes at the top of the clock and a set of lines going across the bottom like sheet music. One major problem I ran into was that one of my music notes had a weird outline where the laser cutter would go over the same line twice, which caused the carboard to catch fire. Once I found a new music note outline I didn't have to worry about it anymore. The clock ended up turning out pretty well.
The second assigment was the regular engraving. We could engrave some design or words on an object like a wristband, a phone case, or a bookmark. I went with the wristband and thought it'd be fun to make it for my friend who loves an internet celebrity names Casey Frey.
The third and final assignment from this cycle was to engrave a design on something round, like a cup or mug. I chose a regular, glass cup and thought it'd look cool with some icicle designs coming down from the lip. I knew it'd take way too long if I had the icicles filled in for the engraving, so I hollowed them out and just had the laser cutter engrave the outline. Unfortunately, at that point it didn't really look like icicles anymore, and they were really faint. It still worked though.


Passcloud
menu
✕
Master Password:
✕
Site Name:
✕
User Profile:
8
12
16
24
32
64
Length:
Year:
Seed:
Password
PIN
USERNAME
PHRASE
NOUN
DICEWARE
Type:
OPTIONS:
⇣
Unique
Alpha
Numeric
Similar
Special
Extended
Yearly
↺
Sync


I am a remote software developer who has worked on projects for State Farm, Discover Card, Jack Daniels, Sloan, Dixon Midland, and a prominent financial institution.
I recently relocated and am looking for chartible organizations that are seeking volunteers. While in Chicago, I mentored students both at Dev Bootcamp and CoderDojoChi. I love volunteering my time and mentoring others to help them improve both technical and soft skills.


Getting start with template is just like ABC, let's get started
- Clone repository from
tailwind-webpack-starter
- Run
npm install or
yarn install
-
For development environment yarn dev
-
For production environment yarn prod content will be placed
in dist directory


Department/School Selection :
Note-1:Before selecting the choice of Ph.D. Programme, applicants are advised to refer to the information brochure (Table 1B, Page No.13) for the seats available in each departments/schools.
(Refer to the Information Brochure for the list of programmes/disciplines/specializations if any and number of seats available in various Departments/Schools)
Note-2:There are no Institute Assistantship available in discipline BC,EC,MS,PH & PY (Annexure-I).
Applicants who wish to apply to these discipline , are advized to contact the concerned departments before applying/submitting the applications. Candidates applying to these discipline should have external fellowship.
PhD. Application Form
{% if user_data.disc_choice1 == "Department of Electrical Engineering" %}
{% endif %}
{% if user_data.disc_choice1 == "Department of Mechanical Engineering" %}
{% endif %}
Would you like to apply for additional Department/School? If yes, then select the Department, and Dicipline if any, from the list provided below. :
Note-:(Please refer the information brochure for the eligibility criteria for applying for additional programmes.)
{% if user_data.disc_choice2 == "Department of Mechanical Engineering" %}
{% endif %}
{% if user_data.disc_choice3 == "Department of Mechanical Engineering" %}
{% endif %}
Whether qualified GATE/GPAT/UGC-CSIR/NET or any other fellowship exam:
Note-1: For P.hD. Programmes GATE/GPAT/CSIR/NET Qualify is mandatory for Institute assistantship.
Note-2: Btech/B.Pharm/Mtech/MPharm graduates from IIT having a CPI or CGPA of 8.0 and above in a 10 point scale are exempted from GATE/GPAT /UGC-CSIR/NET and eligible for Institute assistantship.


Don’t fancy a total-relaxation beach break for your next holiday? Then try your hand at horse trekking, kayaking, yoga, painting and more.
Read More
48 Hours City Breaks
Whether you’re just passing through or making a weekend of it, we give you the 48-hour lowdown on cities across the globe.
Gourmet Traveller
Travel the world in search of the ultimate gourmet destinations discovering the many unique flavours that make up an area’s culinary identity.
Recipes
Inspired dishes that cater for every course and every cuisine.
Cookery Schools
15 Bloomsbury Square, London, WC1A 2LS
Follow our guides to what’s in season with exclusive recipes and features from the country’s top food writers.
Restaurants
We bring you the finest restaurants across the globe.
Food and Travel Reader Awards
Thanks for coming!
Awards
A big thanks to everyone who came to our Awards ceremony and to everyone who voted. Congratulations to all of our winners.
Be sure to pick up a copy of the November issue, in shops 5 October, to see the winners picking up their gongs.
Our Awards
Food
Recipes
Deli of the Month
We bring you the finest Deli's that cater for your every need.
Chefs
Where do the chefs eat when they have the night off and where do they travel to for culinary inspiration?
Travel
Hit the slopes, go on safari or enjoy an adventure cruise, the best trips for those who love a holiday with a bit of action.
Inspiration
From the world’s finest lake stays to the most beautiful Mediterranean islands, for travel inspiration, look no further.
Hotels
We bring you some of the best hotels from around the globe
Drink
Keeping you up to date with the latest wines to try and the best food and wine matches
Wine Opinion
Our wine columnist Adam Lechmere shares his thoughts on wine regions, grape varieties, hot topics and issues.
Food to Wine Matches
Discover what wines to pair with your favourite Food and Travel recipes.
Wine Club
Search and order regional wines through our specialist representatives and take advantage of some expert recipe matches.
Advertisement
Get Premium access to all the latest content online
Subscribe and view full print editions online... Subscribe
Subscribers
Access the full Food and Travel magazine on your tablet, mobile or computer. In addition you will get access to premium content and offers.
View subscription details
Newsletters
Receive exclusive news, offers and competitions direct to your inbox. Sign up to the Food and Travel newsletter.
Submit
The names and logos for Food and Travel Magazine are registered trademarks of Food and Travel Magazine. All text and design is ©2019 · Food and Travel Magazine. All rights reserved.


Note: Clicking "Update" below will not save your changes to the server - it only updates the configuration within the browser, so you'll want to verify your changes and click "Save Changes" when you're ready.


emacs
Tutorial
emacs
is generally easier to pick up than vi
, though this is open to debate.
For the most part, anything typed in will go into the file at the position the cursor is at.
The exception is if one holds down control
(abbreviated as Ctrl
through the rest of this document), meta (usually alt
), or some combination of the two as something is typed in.
These allow for commands to be entered.
There are tons of commands emacs
has built-in, and emacs
can even be extended with custom commands, though for the purposes of this tutorial we will not be getting into any of that.
Generally, for your purposes, there are only a few commands that are necessary, which are described below.
In all these commands, the notation Ctrl-foo
means, “Hold down the control key, and type in foo
while holding down the control key.”
The notation a b
means, “Type in 'a' followed by 'b'.”
emacs
CommandsCtrl-a
: Moves the cursor to the start of the current lineCtrl-e
: Moves the cursor to the end of the current lineShift-Ctrl-,
: Moves the cursor to the start of the current file.
Technically, this command is Ctrl-<
, but shift needs to be held in order to enter <
.
Shift-Ctrl-.
: Moves the cursor to the end of the current file.
Technically, this command is Ctrl->
, but shift needs to be held in order to enter >
.
Ctrl-k
: Delete the text from the cursor extending to the end of the lineCtrl-y
: Paste the last deleted text at the current cursor positionShift-Ctrl--
: Undo the last action.
Technically, this command is Ctrl-_
, but shift needs to be held in order to enter _
.
Ctrl-x s
: Saves the current fileCtrl-xc
: Closes emacs
Ctrl-s foo
: Search for the string “foo
” in the current file.
The search starts from wherever the cursor is currently at, and moves forward through the file, stopping at the end.
Note that it will not wrap through the start of the file, which is the same behavior as all the commands described here.
If you wish to go through the entire file, move the cursor to the start of the file first.
Subsequent presses of Ctrl-s
will jump the cursor to different instances of &ldquofoo
”.
Moving the cursor around (as with the arrow keys) will return behavior to normal, as in, search will no longer be occurring.
Ctrl-r foo
: Much like Ctrl-s foo
, except the search moves backward from the cursor.
This does not mean that it looks for the reverse of “foo
”, but rather that it searches through the part of the file from the start to the cursor, starting closest to the cursor.
Hitting Ctrl-r
again will go through different instances.
Hitting Ctrl-s
will behave as if Ctrl-s foo
had been typed in from the current cursor position.
That is, Ctrl-r
can be used to move backwards through different instances of “foo
”, whereas Ctrl-s
can be used to move forward through different instances of “foo
”.
Meta-x replace-string
: Can be used to find and replace one string with another, automatically.
Information regarding what strings are replaced comes up in a dialog at the bottom of the terminal (referred to as the “minibuffer” in emacs
parlance).
Meta-x query-replace
: Like Meta-x replace-string
, except the user is asked whether they want to replace or not on each match.
This process starts from wherever the cursor is in the file, and stops at the end of the file.
Press 'n
' to skip to the next match (short for “no”), 'y
' to replace the current match (short for “yes”), and Ctrl-g
to cancel.
There are actually more possibilities here than described; hit '?
' to see a listing of them.
Prepared for Computer Science 64 by Kyle Dewey, with inspiration from a similar
vi
tutorial by Prof. Diana Franklin.


The console interface runner is invoked by a command in the form
NUNIT-CONSOLE [inputfiles] [options]
where inputfiles is one or more assemblies or test projects of a type that NUnit can process and options is zero or more options. Input files and options may be mixed in any order.
The console program must always have an assembly or project specified. Assemblies are specified by file name or path, which may be absolute or relative. Relative paths are interpreted based on the current directory.
In addition to assemblies, you may specify any project type that is understood by NUnit. Out of the box, this includes various Visual Studio project types as well as NUnit (.nunit) projects.
If the NUnit V2 framework driver is installed, test assemblies may be run based on any version of the NUnit framework beginning with 2.0. Without the V2 driver, only version 3.0 and higher tests may be run.
| --test=NAMES | Comma-separated list of NAMES of tests to run or explore. This option may be repeated. |
|---|---|
| --include=CATEGORIES | Test CATEGORIES to be included. May be a single category, a comma-separated list of categories or a category expression. |
| --exclude=CATEGORIES | Test CATEGORIES to be excluded. May be a single category, a comma-separated list of categories or a category expression. |
| --config=NAME | NAME of a project configuration to load (e.g.: Debug). |
| --process=PROCESS | PROCESS isolation for test assemblies. Values: Single, Separate, Multiple. If not specified, defaults to Separate for a single assembly or Multiple for more than one. |
| --domain=DOMAIN | DOMAIN isolation for test assemblies. Values: None, Single, Multiple. If not specified, defaults to Separate for a single assembly or Multiple for more than one. |
| --framework=FRAMEWORK | FRAMEWORK type/version to use for tests. Examples: mono, net-4.5, v4.0, 2.0, mono-4.0 |
| --x86 | Run tests in a 32-bit process on 64-bit systems. |
| --dispose-runners | Dispose each test runner after it has finished running its tests |
| --timeout=MILLISECONDS | Set timeout for each test case in MILLISECONDS. |
| --seed=SEED | Set the random SEED used to generate test cases. |
| --workers=NUMBER | Specify the NUMBER of worker threads to be used in running tests. |
| --stoponerror | Stop run immediately upon any test failure or error. |
| --wait | Wait for input before closing console window. |
| --pause | Pause before run to allow debugging. |
| --work=PATH | PATH of the directory to use for output files. |
| --output, --out=PATH | File PATH to contain text output from the tests. |
| --err=PATH | File PATH to contain error output from the tests. |
| --result=SPEC | An output SPEC for saving the test results. This option may be repeated. |
| --explore[=SPEC] | Display or save test info rather than running tests. Optionally provide an output SPEC for saving the test info. This option may be repeated. |
| --noresult | Don't save any test results. |
| --labels=VALUE | Specify whether to write test case names to the output. Values: Off, On, All |
| --trace=LEVEL | Set internal trace LEVEL. Values: Off, Error, Warning, Info, Verbose (Debug) |
| --shadowcopy | Tells .NET to copy loaded assemblies to the shadowcopy directory. |
| --teamcity | Turns on use of TeamCity service messages. |
| --noheader, --noh | Suppress display of program information at start of run. |
| --nocolor, --noc | Displays console output without color. |
| --verbose, -v | Display additional information as the test runs. |
| --help, -h | Display this message and exit. |
By default, this command runs the tests contained in the assemblies and projects specified. If the --explore option is used, no tests are executed but a description of the tests is saved in the specified or default format.
Several options that specify processing of XML output take an output specification as a value. A SPEC may take one of the following forms:
The --result option may use any of the following formats:
The --explore option may use any of the following formats:
If --explore is used without any specification following, a list of test cases is output to the console.
If none of the options {--result, --explore, --noxml} is used, NUnit saves the results to TestResult.xml in nunit3 format.
Any transforms provided must handle input in the native nunit3 format.


Table of Contents
Page 22r
Written Interpretations
Video Interpretations
Speculative Interpretations
Sword and Rotella
Forum Discussions
This play is part of
Giovanni dall'Agocchie
's Three Books on the Art of Defense.
Page 22r
Full text and copyright information:
Wiktenauer
Written Interpretations
Video Interpretations
Speculative Interpretations
Sword and Rotella
Forum Discussions


PM Modi to inaugurate Kartarpur corridor checkpost on November 9
In a statement issued by the Prime Minister's Office said on Friday that before the inauguration ceremony, Modi will pay obeisance at Ber Sahib Gurudwara at Sultanpur Lodhi. He will later participate in a public programme at Dera Baba Nanak.
Prime Minister Narendra Modi will on November 9 inaugurate the integrated check post of the Kartarpur Corridor which will connect the Dera Baba Nanak shrine in India's Punjab with Darbar Sahib at Kartarpur in Narowal district of Pakistan's Punjab province.
In a statement issued by the Prime Minister's Office said on Friday that before the inauguration ceremony, Modi will pay obeisance at Ber Sahib Gurudwara at Sultanpur Lodhi. He will later participate in a public programme at Dera Baba Nanak.
The inauguration of the ICP check post will facilitate, Indian pilgrims to visit to Gurudwara Kartapur Sahib in Pakistan.
India had signed an agreement with the neighbouring nation on October 24 on the modalities for operationalisation of the corridor at 'Zero Point' of the international boundary at Dera Baba Nanak, the statement said.
On November 22, 2018, the Union Cabinet had passed a resolution to celebrate the historic occasion of 550th birth anniversary of Guru Nanak Dev in a "grand and befitting manner", throughout the country and across the globe.
The Cabinet had also approved the building and development of the Kartarpur Sahib Corridor from Dera Baba Nanak to the international boundary, to facilitate pilgrims from India to visit Gurdwara Darbar Sahib Kartarpur, round the year, in a smooth and easy manner.
A state-of-the-art passenger terminal building has been constructed on 15 acres of land. The fully airconditioned building, akin to an airport, has over 50 immigration counters for facilitating about 5,000 pilgrims a day.
It has all the necessary public amenities such as kiosks, washrooms, child care, first aid medical facilities, prayer room and snacks counters inside main building.
The national flag has been hoisted at the site on a 300 feet tall post.


This endpoint triggers notifications. Creating content too quickly using this endpoint may result in abuse rate limiting. See "Abuse rate limits" for details.
POST /repos/:owner/:repo/pulls/:pull_number/reviews
| Name | Type | Description |
|---|---|---|
commit_id |
string |
The SHA of the commit that needs a review. Not using the latest commit SHA may render your review comment outdated if a subsequent commit modifies the line you specify as the position . Defaults to the most recent commit in the pull request when you do not specify a value. |
body |
string |
Required when using REQUEST_CHANGES or COMMENT for the event parameter. The body text of the pull request review. |
event |
string |
The review action you want to perform. The review actions include: APPROVE , REQUEST_CHANGES , or COMMENT . By leaving this blank, you set the review action state to PENDING , which means you will need to submit the pull request review when you are ready. |
comments |
array of draft review comment objects
|
Use the following table to specify the location, destination, and contents of the draft review comment. |
When specifying draft review comments
, you need to include the following values:
| Name | Type | Description |
|---|---|---|
path |
string |
Required. The relative path to the file that necessitates a review comment. |
position |
integer |
Required. The position in the diff where you want to add a review comment. Note this value is not the same as the line number in the file. For help finding the position value, read the note below. |
body |
string |
Required. Text of the review comment. |
Note:
To comment on a specific line in a file, you need to first determine the position of that line in the diff. The GitHub REST API v3 offers the application/vnd.github.v3.diff
media type. To see a pull request diff, add this media type to the Accept
header of a call to the single pull request endpoint.
The position
value equals the number of lines down from the first "@@" hunk header in the file you want to add a comment. The line just below the "@@" line is position 1, the next line is position 2, and so on. The position in the diff continues to increase through lines of whitespace and additional hunks until the beginning of a new file.
{
"commit_id": "ecdd80bb57125d7ba9641ffaa4d7d2c19d3f3091",
"body": "This is close to perfect! Please address the suggested inline change.",
"event": "REQUEST_CHANGES",
"comments": [
{
"path": "file.md",
"position": 6,
"body": "Please add more information here, and fix this typo."
}
]
}
Status: 200 OK
{
"id": 80,
"node_id": "MDE3OlB1bGxSZXF1ZXN0UmV2aWV3ODA=",
"user": {
"login": "octocat",
"id": 1,
"node_id": "MDQ6VXNlcjE=",
"avatar_url": "https://github.com/images/error/octocat_happy.gif",
"gravatar_id": "",
"url": "https://api.github.com/users/octocat",
"html_url": "https://github.com/octocat",
"followers_url": "https://api.github.com/users/octocat/followers",
"following_url": "https://api.github.com/users/octocat/following{/other_user}",
"gists_url": "https://api.github.com/users/octocat/gists{/gist_id}",
"starred_url": "https://api.github.com/users/octocat/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/octocat/subscriptions",
"organizations_url": "https://api.github.com/users/octocat/orgs",
"repos_url": "https://api.github.com/users/octocat/repos",
"events_url": "https://api.github.com/users/octocat/events{/privacy}",
"received_events_url": "https://api.github.com/users/octocat/received_events",
"type": "User",
"site_admin": false
},
"body": "This is close to perfect! Please address the suggested inline change.",
"commit_id": "ecdd80bb57125d7ba9641ffaa4d7d2c19d3f3091",
"state": "CHANGES_REQUESTED",
"html_url": "https://github.com/octocat/Hello-World/pull/12#pullrequestreview-80",
"pull_request_url": "https://api.github.com/repos/octocat/Hello-World/pulls/12",
"_links": {
"html": {
"href": "https://github.com/octocat/Hello-World/pull/12#pullrequestreview-80"
},
"pull_request": {
"href": "https://api.github.com/repos/octocat/Hello-World/pulls/12"
}
}
}


Durt
Durt
Home
About
Manage Garden
Resources
Social
Garden Locator
What are you planting?
ADD PLANT
Please make sure your plant is supported by Durt
Supported Plants
Your Plants
Proceed to Manage Garden!


Collation Formula for Opera, including Lucidarium artis transmutationis metallorum; extracts from the Sommeta and Violetta; and Epistola
Formula 1: 1(7), 2(16), 3(16), 4(16), 5(16), 6(16, -15, -16)
Formula 2: 1(7), 2(16), 3(16), 4(16), 5(16), 6(16, 2 leaves missing after fol. 85)


The HathiTrust Emergency Temporary Access Service (ETAS) allows current BU students, faculty, and staff to have online reading access to selected materials that are in the BU Libraries but are currently unavailable due to the closure of the Libraries' stacks. Faculty, students, and staff can access these materials via BU Libraries Search.
Physical books in BU Libraries that are also available via HathiTrust will have a “Full Text Available at HathiTrust” link included in their records in BU Libraries Search. (Note: This link will also be shown for out-of-copyright items; for those items the link will take you directly to the item without having to follow the steps below.)
After clicking on the "Full Text Available at HathiTrust" link, you'll be prompted for BU Login. After logging in, click the temporary access link.
...that will lead to a page with a Check Out button.
Books can be “checked out” to read online but cannot be downloaded. The checkout time is one hour. Access will renew automatically at the end of that hour, unless another user requests the book after that hour is over. The number of users who can check out a book at a time is based on how many physical copies of the book the Libraries have.
NOTE: If a book that is supposed to be available does not show the Temporary access link, log out of HathiTrust and log back in. (BU log-ins expire, but HathiTrust may not show that.)