Once the NTP software distribution has been compiled and installed and the configuration file constructed, the next step is to verify correct operation and fix any bugs that may result. Usually, the command line that starts the daemon is included in the system startup file, so it is executed only at system boot time; however, the daemon can be stopped and restarted from root at any time. Usually, no command-line arguments are required, unless special actions described in the xntpd.8 man page are required. Once started, the daemon will begin sending messages, as specified in the configuration file, and interpreting received messages.
The best way to verify correct operation is using the ntpq
and
xntpdc
utility programs, either on the server itself
or from another machine elsewhere in the network. The ntpq
program implements the management functions specified in Appendix A of
the NTP specification RFC-1305,
Appendix A . The xntpdc
program implements additional
functions not provided in the standard. Both programs can be used to
inspect the state variables defined in the specification and, in the
case of xntpdc
, additional ones of interest. In addition,
the xntpdc
program can be used to selectively enable and
disable some functions of the daemon while the daemon is running.
In extreme cases with elusive bugs, the daemon can operate in two
modes, depending on the presence of the -d
command-line
debug switch. If not present, the daemon detaches from the controlling
terminal and proceeds autonomously. If one or more -d
switches are present, the daemon does not detach and generates special
output useful for debugging. In general, interpretation of this output
requires reference to the sources.
Some problems are immediately apparent when the daemon first starts
running. The most common of these are the lack of a ntp (UDP port 123)
in the host /etc/services
file. Note that NTP does not use
TCP in any form. Other problems are apparent in the system log file. The
log file should show the startup banner, some cryptic initialization
data, and the computed precision value. The next most common problem is
incorrect DNS names. Check that each DNS name used in the configuration
file responds to the Unix ping
command.
When first started, the daemon normally polls the servers listed in
the configuration file at 64-second intervals. In order to allow a
sufficient number of samples for the NTP algorithms to reliably
discriminate between correctly operating servers and possible intruders,
at least four valid messages from at least one server is required before
the daemon can not set the local clock. However, if the current local
time is greater than 1000 seconds in error from the server time, the
daemon will not set the local clock; instead, it will plant a message in
the system log and shut down. It is necessary to set the local clock to
within 1000 seconds first, either by a time-of-year hardware clock, by
first using the ntpdate
program or manually be eyeball and wristwatch.
After starting the daemon, run the ntpq
program using
the -n
switch, which will avoid possible distractions due
to name resolution problems. Use the pe
command to display
a billboard showing the status of configured peers and possibly other
clients poking the daemon. After operating for a few minutes, the
display should be something like:
ntpq>pe remote refid st when poll reach delay offset disp ======================================================================== +<IP_ADDRESS> <IP_ADDRESS> 2 131 256 373 9.89 16.28 23.25 *<IP_ADDRESS> .WWVB. 1 137 256 377 280.62 21.74 20.23 -<IP_ADDRESS> <IP_ADDRESS> 2 49 128 376 294.14 5.94 17.47 +<IP_ADDRESS> .WWVB. 1 173 256 377 279.95 20.56 16.40
The host addresses shown in the remote
column should
agree with the DNS entries in the configuration file, plus any peers not
mentioned in the file at the same or lower than your stratum that happen
to be configured to peer with you. Be prepared for surprises in cases
where the peer has multiple addresses or multiple names. The
refid
entry shows the current source of synchronization for
each peer, while the st
reveals its stratum and the
poll
entry the polling interval, in seconds. The
when
entry shows the time since the peer was last heard,
normally in seconds, while the reach
entry shows the status
of the reachability register (see RFC-1305), which is in octal format.
The remaining entries show the latest delay, offset and dispersion
computed for the peer, in milliseconds.
The tattletale character at the left margin displays the
synchronization status of each peer. The currently selected peer is
marked *
, while additional peers designated acceptable for
synchronization, but not currently selected, are marked +
.
Peers marked *
and +
are included in a
weighted average computation to set the local clock; the data produced
by peers marked with other symbols are discarded. See the
ntpq
documentation for the meaning of these symbols.
Additional details for each peer separately can be determined by the
following procedure. First, use the as
command to display
an index of association identifiers, such as
ntpq>as ind assID status conf reach auth condition last_event cnt =========================================================== 1 11670 7414 no yes ok synchr. reachable 1 2 11673 7614 no yes ok sys.peer reachable 1 3 11833 7314 no yes ok outlyer reachable 1 4 11868 7414 no yes ok synchr. reachable 1
Each line in this billboard is associated with the corresponding line
the pe
billboard above. Next, use the rv
command and the respective identifier to display a detailed synopsis of
the selected peer, such as
ntpq>rv 11670 status=7414 reach, auth, sel_sync, 1 event, event_reach srcadr=<IP_ADDRESS>, srcport=123, dstadr=<IP_ADDRESS>, dstport=123, keyid=1, stratum=2, precision=-10, rootdelay=362.00, rootdispersion=21.99, refid=<IP_ADDRESS>, reftime=af00bb44.849b0000 Fri, Jan 15 1993 4:25:40.517, delay= 9.89, offset= 16.28, dispersion=23.25, reach=373, valid=8, hmode=2, pmode=1, hpoll=8, ppoll=10, leap=00, flash=0x0, org=af00bb48.31a90000 Fri, Jan 15 1993 4:25:44.193, rec=af00bb48.305e3000 Fri, Jan 15 1993 4:25:44.188, xmt=af00bb1e.16689000 Fri, Jan 15 1993 4:25:02.087, filtdelay= 16.40 9.89 140.08 9.63 9.72 9.22 10.79 122.99, filtoffset= 13.24 16.28 -49.19 16.04 16.83 16.49 16.95 -39.43, filterror= 16.27 20.17 27.98 31.89 35.80 39.70 43.61 47.52
A detailed explanation of the fields in this billboard are beyond the scope of this discussion; however, most variables defined in the specification RFC-1305 can be found. The most useful portion for debugging is the last three lines, which give the roundtrip delay, clock offset and dispersion for each of the last eight measurement rounds, all in milliseconds. Note that the dispersion, which is an estimate of the error, increases as the age of the sample increases. From these data, it is usually possible to determine the incidence of severe packet loss, network congestion, and unstable local clock oscillators. There are no hard and fast rules here, since every case is unique; however, if one or more of the rounds show zeros, or if the clock offset changes dramatically in the same direction for each round, cause for alarm exists.
Finally, the state of the local clock can be determined using the
rv
command (without the argument), such as
ntpq>rv status=0664 leap_none, sync_ntp, 6 events, event_peer/strat_chg system="UNIX", leap=00, stratum=2, rootdelay=280.62, rootdispersion=45.26, peer=11673, refid=<IP_ADDRESS>, reftime=af00bb42.56111000 Fri, Jan 15 1993 4:25:38.336, poll=8, clock=af00bbcd.8a5de000 Fri, Jan 15 1993 4:27:57.540, phase=21.147, freq=13319.46, compliance=2
The most useful data in this billboard show when the clock was last
adjusted reftime
, together with its status and most recent
exception event. An explanation of these data is in the specification
RFC-1305.
When nothing seems to happen in the pe
billboard after
some minutes, there may be a network problem. The most common network
problem is an access controlled router on the path to the selected peer.
No known public NTP time server selectively restricts access at this
time, although this may change in future; however, many private networks
do. It also may be the case that the server is down or running in
unsynchronized mode due to a local problem. Use the ntpq
program to spy on its own variables in the same way you can spy on your
own.
Once the daemon has set the local clock, it will continuously track the discrepancy between local time and NTP time and adjust the local clock accordingly. There are two components of this adjustment, time and frequency. These adjustments are automatically determined by the clock discipline algorithm, which functions as a hybrid phase/frequency feedback loop. The behavior of this algorithm is carefully controlled to minimize residual errors due to network jitter and frequency variations of the local clock hardware oscillator that normally occur in practice. However, when started for the first time, the algorithm may take some time to converge on the intrinsic frequency error of the host machine.
It has sometimes been the experience that the local clock oscillator
frequency error is too large for the NTP discipline algorithm, which can
correct frequency errors as large as 30 seconds per day. There are two
possibilities that may result in this problem. First, the hardware time-
of-year clock chip must be disabled when using NTP, since this can
destabilize the discipline process. This is usually done using the tickadj
program and the
-s
command line argument, but other means may be necessary.
For instance, in the Sun Solaris kernel, this must be done using a
command in the system startup file.
Normally, the daemon will adjust the local clock in small steps in such a way that system and user programs are unaware of its operation. The adjustment process operates continuously as long as the apparent clock error exceeds 128 milliseconds, which for most Internet paths is a quite rare event. If the event is simply an outlyer due to an occasional network delay spike, the correction is simply discarded; however, if the apparent time error persists for an interval of about 20 minutes, the local clock is stepped to the new value (as an option, the daemon can be compiled to slew at an accelerated rate to the new value, rather than be stepped). This behavior is designed to resist errors due to severely congested network paths, as well as errors due to confused radio clocks upon the epoch of a leap second.
If the ntpq
or xntpdc
programs do not
show that messages are being received by the daemon or that received
messages do not result in correct synchronization, verify the following:
/etc/services
file host machine is
configured to accept UDP packets on the NTP port 123. NTP is
specifically designed to use UDP and does not respond to TCP.
xntpd
messages about
configuration errors, name-lookup failures or initialization problems.
xntpdc
program and iostats
command, verify that the received packets and packets sent counters are
incrementing. If the packets send counter does not increment and the
configuration file includes designated servers, something may be wrong
in the network configuration of the xntpd host. If this counter does
increment and packets are actually being sent to the network, but the
received packets counter does not increment, something may be wrong in
the network or the server may not be responding.
rec
timestamp in the
pe
billboard shows a date in 1972, received packets are
probably being discarded for some reason. There is a handy, undocumented
state variable flash
visible in the
pe
billboard. The value is in hex and normally has the value
zero (OK). However, if something is wrong, the bits of this variable,
reading from the right, correspond to the sanity checks listed in
Section 3.4.3 of the NTP specification
RFC-1305. A bit other than zero indicates the associated sanity
check failed.
org, rec
and xmt
timestamps in the pe
billboard appear current, but the
local clock is not set, as indicated by a stratum number less than 16 in
the rv
command without arguments, verify that valid clock
offset, roundtrip delay and dispersion are displayed for at least one
peer. The clock offset should be less than 1000 seconds, the roundtrip
delay less than one second and the dispersion less than one second.
xntpdc
program (or temporary configuration file) and
disable pll
command to prevent the xntpd
daemon from setting the clock. Using the ntpq
or
xntpdc
programs, watch the apparent offset as it varies
over time to determine the intrinsic frequency error. If the error
increases by more than 22 milliseconds per 64-second poll interval, the
intrinsic frequency must be reduced by some means. The easiest way to do
this is with the tickadj
program
and the -t
command line argument.


HTML5 Document Structure Example
This page should be tried in safari, chrome or Mozila.
HTML Tutorial
CSS Tutorial
JavaScript Tutorial
Once article can have multiple sections


---
layout: archive
title: Lean Prover Zulip Chat Archive
permalink: 113489newmembers/54434Beginnerquestioninequality.html
---
The archive URL format has changed. Please find this page at:
https://leanprover-community.github.io/archive/stream/113489-new-members/topic/Beginner.20question.3A.20inequality.html


*** User 2017-04-20 14:15:21.015 ***
14:15:21.015 [info] Disabling blocking sync...
*** User 2017-04-20 14:15:21.018 ***
14:15:21.018 [info] Configured peer_service_manager partisan_default_peer_service_manager on node 'node-1@antoine-PC'
*** User 2017-04-20 14:15:22.357 ***
14:15:22.357 [info] Enabling membership...
*** User 2017-04-20 14:15:22.358 ***
14:15:22.358 [info] Disabling blocking sync...
*** User 2017-04-20 14:15:22.361 ***
14:15:22.361 [info] Configured peer_service_manager partisan_default_peer_service_manager on node 'node-2@antoine-PC'
*** User 2017-04-20 14:15:23.688 ***
14:15:23.688 [info] Enabling membership...
*** User 2017-04-20 14:15:23.689 ***
14:15:23.689 [info] Disabling blocking sync...
*** User 2017-04-20 14:15:23.692 ***
14:15:23.692 [info] Configured peer_service_manager partisan_default_peer_service_manager on node 'node-3@antoine-PC'
*** User 2017-04-20 14:15:23.852 ***
Joining node: 'node-3@antoine-PC' to 'runner@antoine-PC' at port 56210
*** User 2017-04-20 14:15:23.852 ***
Joining issued: 'node-3@antoine-PC' to 'runner@antoine-PC' at port 56210
*** User 2017-04-20 14:15:23.852 ***
Joining node: 'node-2@antoine-PC' to 'runner@antoine-PC' at port 55416
*** User 2017-04-20 14:15:23.853 ***
Joining issued: 'node-2@antoine-PC' to 'runner@antoine-PC' at port 55416
*** User 2017-04-20 14:15:23.853 ***
Joining node: 'node-1@antoine-PC' to 'runner@antoine-PC' at port 55805
*** User 2017-04-20 14:15:23.853 ***
Joining issued: 'node-1@antoine-PC' to 'runner@antoine-PC' at port 55805
*** User 2017-04-20 14:15:23.914 ***
Waiting for join to complete with manager: partisan_default_peer_service_manager
*** User 2017-04-20 14:15:23.915 ***
Finished.
*** CT 2017-04-20 14:15:23.915 *** COVER INFO
Adding nodes to cover test: ['runner@antoine-PC']
*** CT 2017-04-20 14:15:23.920 *** COVER INFO
Successfully added nodes to cover test: []
*** User 2017-04-20 14:15:31.948 ***
Case finished: map_test
*** User 2017-04-20 14:15:38.975 ***
14:15:38.975 [info] Application lasp exited with reason: stopped
*** User 2017-04-20 14:15:38.976 ***
14:15:38.976 [info] Application partisan exited with reason: stopped
=== Ended at 2017-04-20 14:15:38
=== successfully completed test case
=== Returned value: ok
Test run history
| Top level test index


class: center, middle, inverse, title-slide # Introduction to Statistics ## Part II: Introduction to Statistical Inference ### Stéphane Guerrier & Yuming Zhang ### 8 January 2021 --- # Review - .smallest[.pink[Statistical inference] allows to infer the properties of a population based on an observed sample.] - .smallest[For example, .pink[proportion estimation]] `\(\color{#e64173}{\hat{p}}\)` .smallest[is .purple[random] as it is sample dependent. Therefore, we need to assess the .purple[uncertainty] of] `\(\hat{p}\)` .smallest[to infer on] `\(p\)`. - .smallest[One common approach to measure uncertainty is to use .pink[confidence intervals], which rely on .pink[Central Limit Theorem]. In the case of proportion estimation, we have] `$$\small \hat{p} \overset{\cdot}{\sim} \mathcal{N}\Bigg(\color{#e64173}{p}, \color{#6A5ACD}{\frac{p(1-p)}{n}}\Bigg).$$` - .smallest[Confidence intervals correspond to a range of values that include the population value with .purple[a certain level of confidence]. The] `\(1-\alpha\)` .smallest[confidence interval for] `\(p\)` .smallest[is given by] `\(\color{#e64173}{\hat{p}} \pm Z_{1-\alpha/2}\color{#6A5ACD}{\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}}\)` .smallest[, and obviously, it is random.] - .smallest[In Statistics, a decision or prediction can never be made with certainty. There always exists .purple[a trade-off between the risk and the precision of the conclusion], and how much risk is acceptable depends on the context.] --- # How to test a (scientific) hypothesis? .center[ .turquoise["In god we trust, all others must bring data." .smallest[👋]] ] - .smallest[To assess the .pink[validity of a (scientific) hypothesis], the scientific community generally agrees on .purple[a specific procedure].] - .smallest[These hypotheses can be .pink[nearly anything], such as:] 1. .smallest[Coffee consumption increases blood pressure. ☕] 2. .smallest[Republican politicians are bad/good for the American Economy. ] 3. .smallest[A glass of red wine is as good as an hour at the gym. 🍷🏋] - .smallest[This procedure involves the design of an experiment and then the collection of data to compute a metric (called p-value) which .purple[evaluates the adequacy between the data and your original hypothesis.]] - .smallest[There is generally .pink[a specific threshold] (typically 5%), and if the p-value falls below this threshold .purple[we can claim that we have statistically significant result(s) validating our hypothesis.]] .footnote[.smallest[👋 From W. Edwards Deming]] --- # Statistics vs Truth - .smallest[.pink[Statistically significant results are not necessarily the truth], as there isn't a threshold (again typically 5%) that separates real results from the false ones.] - .smallest[This procedure simply provides us with one piece of a puzzle that should be considered in the context of other evidence.]  .footnote[.smallest[👋] Read the original article: "*This is why you shouldn't believe that exciting new medical study*" [here](https://www.vox.com/2015/3/23/8264355/research-study-hype).] --- # A hypothesis testing - In the Biden-Trump example, recall that we want to verify if Biden will win the popular vote. Then we can translate this into the following hypotheses: .center[ `\(H_0: p = 0.5\)` and `\(H_a: p > 0.5\)`. ] - In a hypothesis test, the statement being tested is called the .hi-pink[null hypothesis] `\(\color{#e64173}{H_0}\)`. A hypothesis test is designed to assess the strength of the evidence against the null hypothesis. - The .hi-pink[alternative hypothesis] `\(\color{#e64173}{H_a}\)` is the statement we hope or suspect to be true instead of `\(H_0\)`. - Each hypothesis .purple[excludes the other], so that one can .purple[exclude one in favor of the other] using the data. --- # A hypothesis testing | | `\(H_0\)` is true | `\(H_0\)` is false | | ------------------- |---------------------------------------------| ----------------------------------------| | Can't reject `\(H_0\)` | `\(\text{Correct decision (prob=}1-\alpha)\)` | `\(\text{Type II error (prob=}1-\beta)\)` | | Reject `\(H_0\)` | `\(\text{Type I error (prob=}\alpha)\)` | `\(\text{Correct decision (prob=}\beta)\)` | - If we reject `\(H_0\)` when in fact `\(H_0\)` is true, this is a .pink[type I error] (also called .pink[false positive]). If we accept `\(H_0\)` when in fact `\(H_a\)` is true, this is a .purple[type II error] (also called .purple[false negative]). - A test is of .pink[significance level] `\(\color{#e64173}{\alpha}\)` when the probability to make a type I error is `\(\alpha\)`. Usually we consider `\(\alpha = 5\%\)`, however, this can vary depending on the context. - A test is of .purple[power] `\(\color{#6A5ACD}{\beta}\)` when the probability to make a type II error is `\(1-\beta.\)` In other words, the power of a test is its probability to reject `\(H_0\)` when `\(H_0\)` is false. --- # Test statistics and P-values - .smallest[A hypothesis testing is based on a .hi-pink[test statistic], which measures the difference between the sample estimate and the hypothesized value in terms of its standard deviation, i.e.] `$$\small \text{test statistic} = \frac{\text{sample estimate - hypothesized value}}{\text{standard deviation of sample estimate}}.$$` - .smallest[For example, consider a test for a single proportion with] `\(\small H_0: p = p_0\)` .smallest[and] `\(\small H_a: p > p_0\)`.smallest[, the corresponding test statistic can be computed as] `$$Z = \frac{\hat{p}-\color{#e64173}{p_0}}{\color{#6A5ACD}{\sqrt{\frac{p_0(1-p_0)}{n}}}} \overset{\cdot}{\sim} \mathcal{N}(0,1).$$` - .smallest[The .hi-purple[p-value] is defined as the probability, assuming that] `\(\small H_0\)` .smallest[is true, that the test statistic will take a value at least as extreme as that actually observed.] 🤯😱 - .smallest[Informally, .pink[a p-value can be understood as a measure of plausibility of the null hypothesis given the data]. Small p-value indicates strong evidence against] `\(\small H_0\)`. --- # How to understand p-values? - When the p-value is small enough (i.e. smaller than the significance level `\(\alpha\)`), one says that the test based on the null and alternative hypotheses is .pink[significant] or that the null hypothesis is rejected in favor of the alternative. .purple[This is generally what we want because it "verifies" our (research) hypothesis]. - When the p-value is not small enough, with the available data, we cannot reject the null hypothesis so .pink[nothing] can be concluded. 🤔 - With a sample of data, the obtained p-value summarizes somehow the .pink[incompatibility between the data and the model] (random process) constructed under the set of assumptions. .center[ .purple["Absence of evidence is not evidence of absence." .smallest[👋]] ] .footnote[.smallest[👋] From the British Medical Journal.] --- # How to understand p-values?  👋 .smallest[If you want to know more have a look [here](https://xkcd.com/1478/).] --- # P-values may be controversial .purple[P-values have been misused] many times because understanding what they mean is not intuitive!

 👋 .smallest[If you want to know more have a look [here](https://fivethirtyeight.com/features/statisticians-found-one-thing-they-can-agree-on-its-time-to-stop-misusing-p-values/).] --- # Test for a single proportion: Example 1 .panelset[ .panel[.panel-name[Problem] .smallest[Returning to our Biden-Trump example, suppose we believe (or hope to show that) .pink[Biden will have more than 50% of the votes]. We collect data with] `\(\small n = 600\)` .smallest[and] `\(\small m = 322\)`.smallest[. We will consider the following steps to set up the test:] 1. .smallest[.purple[Define hypotheses:]] `\(\small H_0: p = 0.5\)` .smallest[and] `\(\small H_a: p \color{#e64173}{>} 0.5\)`. 2. .smallest[.purple[Define]] `\(\small \color{#6A5ACD}{\alpha}\)`.smallest[: We consider] `\(\small \alpha = 5\%\)` .smallest[(as it is generally a default value 🤔).] 3. .smallest[.purple[Compute p-value]: p-value = ] `\(\small 3.959\%\)` .smallest[(see R code tab for details).] 4. .smallest[.purple[Conclusion:] We have p-value <] `\(\small \alpha\)` .smallest[so we can reject the null hypothesis and conclude that the proportion of voters for Biden is greater than 50% at the significance level of 5%.] ] .panel[.panel-name[`R` Code ] ```r prop.test(x = 322, n = 600, p = 0.5, alternative = "greater") ``` ``` #> #> 1-sample proportions test with continuity correction #> #> data: 322 out of 600, null probability 0.5 #> X-squared = 3.0817, df = 1, p-value = 0.03959 #> alternative hypothesis: true p is greater than 0.5 #> 95 percent confidence interval: #> 0.5022582 1.0000000 #> sample estimates: #> p #> 0.5366667 ``` ] ] --- # Test for a single proportion: Example 1 .panelset[ .panel[.panel-name[Problem] .smallest[What if we want to check if .pink[Trump will have more than 50% of the votes]? Using the same data, we set up the test as follows:] 1. .smallest[.purple[Define hypotheses:]] `\(\small H_0: p = 0.5\)` .smallest[and] `\(\small H_a: p \color{#e64173}{<} 0.5\)`. 2. .smallest[.purple[Define]] `\(\small \color{#6A5ACD}{\alpha}\)`.smallest[: We consider] `\(\small \alpha = 5\%\)`.smallest[.] 3. .smallest[.purple[Compute p-value]: p-value = ] `\(\small 96.04\%\)` .smallest[(see R code tab for details).] 4. .smallest[.purple[Conclusion:] We have p-value >] `\(\small \alpha\)` .smallest[and we cannot reject the null hypothesis. We don't have enough evidence to conclude that Trump will have more than 50% of the votes at the significance level of 5%.] ] .panel[.panel-name[`R` Code ] ```r prop.test(x = 322, n = 600, p = 0.5, alternative = "less") ``` ``` #> #> 1-sample proportions test with continuity correction #> #> data: 322 out of 600, null probability 0.5 #> X-squared = 3.0817, df = 1, p-value = 0.9604 #> alternative hypothesis: true p is less than 0.5 #> 95 percent confidence interval: #> 0.0000000 0.5707377 #> sample estimates: #> p #> 0.5366667 ``` ] ] --- # Test for a single proportion: Example 2 .panelset[ .panel[.panel-name[Problem] .smallest[A researcher who is studying the effects of income levels on breastfeeding of infants hypothesizes that .pink[countries where the income level is lower have a higher rate of infant breastfeeding than higher income countries]. It is .hi.purple[known] that in Germany (high-income country), 22% of all babies are breastfeed. In Tajikistan (low-income country) researchers found that in a random sample of 500 new mothers that 125 were breastfeeding their infants. What can we conclude?] 1. .smallest[.purple[Define hypotheses:]] `\(\small H_0: p = 0.22\)` .smallest[and] `\(\small H_a: p \color{#e64173}{>} 0.22\)`. 2. .smallest[.purple[Define]] `\(\small \color{#6A5ACD}{\alpha}\)`.smallest[: We consider] `\(\small \alpha = 5\%\)`.smallest[.] 3. .smallest[.purple[Compute p-value]: p-value = ] `\(\small 5.874\%\)` .smallest[(see R code tab for details).] 4. .smallest[.purple[Conclusion:] We have p-value >] `\(\small \alpha\)` .smallest[and we cannot reject the null hypothesis. We don't have enough evidence to conclude, at the significance level of 5%, that countries where the income level is lower have a higher rate of infant breastfeeding than higher income countries.] ] .panel[.panel-name[`R` Code ] ```r prop.test(x = 125, n = 500, p = 0.22, alternative = "greater") ``` ``` #> #> 1-sample proportions test with continuity correction #> #> data: 125 out of 500, null probability 0.22 #> X-squared = 2.4505, df = 1, p-value = 0.05874 #> alternative hypothesis: true p is greater than 0.22 #> 95 percent confidence interval: #> 0.218598 1.000000 #> sample estimates: #> p #> 0.25 ``` ] ] --- # The one-sample Student's t-test - .smallest[So far we discuss about significance test for a single proportion. How can we perform a .purple[test for the mean of a population]? We will use .hi-pink[the Student's t-test] (or simply t-test) .smallest[👋], in which the test statistic follows a .pink[Student's t distribution] under the null hypothesis.] - .smallest[The Student's t distributions were discovered in 1908 by William S. Gosset, who is a statistician employed by the Guinness brewing company. Gosset devised the t-test as an economical way to monitor the quality of stout. At that time, the company forbade its scientists from publishing their findings, so Gosset published his statistical work under the pen name "Student" in the journal Biometrika. So the Student's t-test is named after Gosset's contributions. 🍻🤓] - .smallest[A t-test is useful to test if the population mean is .pink[greater, smaller, or different than] a hypothesized value.] .footnote[.smallest[👋 The Student's t-test is used when the population variance is unknown. In some uncommon cases where the population variance is known, we use the Z-test, where the test statistic follows a standard normal distribution. Check out more [here](https://en.wikipedia.org/wiki/Z-test).]] --- # Data visualization - .smallest[Before assessing if the mean of a population fits our hypothesis, we generally .purple[start by presenting our data visually].] - .smallest[A standard method is to consider a .hi-pink[boxplot].]  .footnote[.smallest[👋 More information about boxplot can be found [here](http://web.pdx.edu/~stipakb/download/PA551/boxplot.html).]] --- # The one-sample Student's t-test - .smaller[Consider the following example where we wish to assess .pink[if the population means of some of the groups below are larger than 0.]]  --- # The one-sample t-test: Example 1 .panelset[ .panel[.panel-name[Problem] .smallest[The Nielsen Company, a global information and media company, announced that the U.S. cell phone subscribers spend 5.4 hours on average per week in watching videos on their phones. We want to know whether the U.S. college students .pink[spend different time watching videos on their phones than the average] with the following sample (in hours) of size 8: `$$11.9 \;\; 2.8 \;\; 3.0 \;\; 6.2 \;\; 4.7 \;\; 9.8 \;\; 11.1 \;\; 7.8.$$`] 1. .smallest[.purple[Define hypotheses:]] `\(\small H_0: \mu = 5.4\)` .smallest[and] `\(\small H_a: \mu \color{#e64173}{\neq} 5.4\)`. 2. .smallest[.purple[Define]] `\(\small \color{#6A5ACD}{\alpha}\)`.smallest[: We consider] `\(\small \alpha = 5\%\)`.smallest[.] 3. .smallest[.purple[Compute p-value]: p-value = ] `\(\small 20.4\%\)` .smallest[(see R code tab for details).] 4. .smallest[.purple[Conclusion:] We have p-value >] `\(\small \alpha\)` .smallest[and we cannot reject the null hypothesis at the significance level of 5%. We don't have enough evidence to conclude that the U.S. college students spend different time watching videos on their phones than the average.] ] .panel[.panel-name[Boxplot Code] ```r data = c(11.9, 2.8, 3.0, 6.2, 4.7, 9.8, 11.1, 7.8) boxplot(data, col = "lightgrey", main = "U.S. college students", ylab = "Time watching videos (week)") points(1, mean(data), pch = 16, col = 1) abline(h = 5.4, lwd = 1, lty = 2) ``` ] .panel[.panel-name[Boxplot]  ] .panel[.panel-name[`R` Code ] ```r data = c(11.9, 2.8, 3.0, 6.2, 4.7, 9.8, 11.1, 7.8) t.test(x = data, mu = 5.4, alternative = "two.sided") ``` ``` #> #> One Sample t-test #> #> data: data #> t = 1.4008, df = 7, p-value = 0.204 #> alternative hypothesis: true mean is not equal to 5.4 #> 95 percent confidence interval: #> 4.187213 10.137787 #> sample estimates: #> mean of x #> 7.1625 ``` ] ] --- # The one-sample t-test: Example 2 .panelset[ .panel[.panel-name[Problem] An investor sues his broker and brokerage firm because he thinks his stock portfolio .pink[performs worse than the S&P 500 monthly average return rate, 0.95%]. We collect the rates of return (in %) for 10 months when the account was managed by the broker: `$$\small -2.36 \;\; -1.82 \;\; 0.46 \;\; 0.65 \;\; -2.14 \;\; -1.63 \;\; -25.5 \;\; 0.25 \;\; -4.34 \;\; 0.91$$` Please perform the t-test following the steps and what conclusion can you draw? 🤔 ] .panel[.panel-name[Boxplot Code] ```r data = c(-2.36, -1.82, 0.46, 0.65, -2.14, -1.63, -25.5, 0.25, -4.34, 0.91) boxplot(data, col = "lightgrey", main = "Stock portfolio", ylab = "Monthly rates of return (%)") points(1, mean(data), pch = 16, col = 1) abline(h = 0.95, lwd = 1, lty = 2) ``` ] .panel[.panel-name[Boxplot]  ] .panel[.panel-name[Solution] 1. .purple[Define hypotheses:] `\(H_0: \mu = 0.95\)` and `\(H_a: \mu \color{#e64173}{<} 0.95\)`. 2. .purple[Define] `\(\color{#6A5ACD}{\alpha}\)`: We consider `\(\alpha = 5\%\)`. 3. .purple[Compute p-value]: p-value = `\(5.237\%\)` (see R code tab for details). 4. .purple[Conclusion:] We have p-value > `\(\alpha\)` and we cannot reject the null hypothesis at the significance level of 5%. We don't have enough evidence to conclude that the investor's stock portfolio performs worse than the S&P 500 monthly average return rate. 🙀 ] .panel[.panel-name[`R` Code ] ```r data = c(-2.36, -1.82, 0.46, 0.65, -2.14, -1.63, -25.5, 0.25, -4.34, 0.91) t.test(x = data, mu = 0.95, alternative = "less") ``` ``` #> #> One Sample t-test #> #> data: data #> t = -1.8039, df = 9, p-value = 0.05237 #> alternative hypothesis: true mean is less than 0.95 #> 95 percent confidence interval: #> -Inf 1.02288 #> sample estimates: #> mean of x #> -3.552 ``` ] ] --- # Limitations of the one-sample t-test .smallest[For the t-test to work, the data is necessary to satisfy the followings:] - .smallest[There are .pink[no outliers];] - .smallest[The sample distribution is at least .pink[approximately normal] with no strong skewness.] .smallest[Therefore, before proceeding to any inference, we should check the data preliminarily using .purple[boxplot] or .purple[histogram] or .purple[QQ plot]] .smallest[👋] .smallest[to see if a t-test can be used.] .pull-left[  ] .pull-right[  ] .footnote[.smallest[👋 Check out [QQ plot](https://en.wikipedia.org/wiki/Q%E2%80%93Q_plot).]] --- # One-sample Wilcoxon signed rank test - .smallest[What can we do if there are outliers in the data and/or if the population distribution is clearly not normal, especially when the sample size is very small? We use the .hi-pink[Wilcoxon signed rank test]!] - .smallest[The .purple[rank] is the position index of each observation when we order them from smallest to largest, starting with rank 1 for the smallest observation. ] - .smallest[The Wilcoxon signed rank test only uses the ranks (i.e. the ordering) of the observations, and makes no use of their actual numerical values. Therefore, it is a .turquoise[nonparametric test].] - .smallest[The Wilcoxon signed rank test depends on the .purple[Wilcoxon signed rank statistic], which is the sum of the ranks of .purple[only the positive] values (or .purple[only the negative] values).] - .smallest[Unlike the one-sample t-test whose hypotheses are on the population mean, the Wilcoxon signed rank test states the hypotheses in terms of .pink[population median]. When the distribution is symmetric, the mean and the median of the population have the same value.] --- # Revisit the portfolio example .panelset[ .panel[.panel-name[Problem] .smallest[We recall the portfolio example in which we want to test if the investor's stock portfolio .pink[performs worse than the S&P 500 monthly average return rate, 0.95%]. We recall the sample as follows:] `$$\small -2.36 \;\; -1.82 \;\; 0.46 \;\; 0.65 \;\; -2.14 \;\; -1.63 \;\; -25.5 \;\; 0.25 \;\; -4.34 \;\; 0.91$$` .smallest[Let's perform the Wilcoxon signed rank test and see if we draw different conclusion than the one-sample t-test?] 1. .smallest[.purple[Define hypotheses:]] `\(\small H_0: \text{median} = 0.95\)` .smallest[and] `\(\small H_a: \text{median} \color{#e64173}{<} 0.95\)`.smallest[.] 2. .smallest[.purple[Define]] `\(\small \color{#6A5ACD}{\alpha}\)`.smallest[: We consider] `\(\small \alpha = 5\%\)`.smallest[.] 3. .smallest[.purple[Compute p-value]: p-value =] `\(\small 0.09766\%\)` .smallest[(see R code tab for details).] 4. .smallest[.purple[Conclusion:] We have p-value <] `\(\small \alpha\)` .smallest[so we can reject the null hypothesis at the significance level of 5% and conclude that the investor's stock portfolio performs worse than the S&P 500 average. 🥳] ] .panel[.panel-name[Boxplot Code] ```r data = c(-2.36, -1.82, 0.46, 0.65, -2.14, -1.63, -25.5, 0.25, -4.34, 0.91) boxplot(data, col = "lightgrey", main = "Stock portfolio", ylab = "Monthly rates of return (%)") points(1, mean(data), pch = 16, col = 1) abline(h = 0.95, lwd = 1, lty = 2) ``` ] .panel[.panel-name[Boxplot]  ] .panel[.panel-name[`R` Code ] ```r data = c(-2.36, -1.82, 0.46, 0.65, -2.14, -1.63, -25.5, 0.25, -4.34, 0.91) wilcox.test(x = data, mu = 0.95, alternative = "less") ``` ``` #> #> Wilcoxon signed rank test #> #> data: data #> V = 0, p-value = 0.0009766 #> alternative hypothesis: true location is less than 0.95 ``` ] ] --- # Summary - .pink[In general, assessing whether the data is normally distributed is very difficult.] In fact, there is no satisfactory statistical method for it. - In practice, we recommend to always first look at the boxplot, assess the symmetry of the data and verify that there is no outlier (or data that is too far from the bulk of the data). If these conditions do not appear to be satisfied, it is preferable to use the one-sample Wilcoxon signed rank test. - .purple[Our recommendation is to always use the one-sample Wilcoxon signed rank test], as it is generally challenging to assess if the assumptions are satisfied for a t-test, and if the data actually satisfy the normality condition. We may lose a bit of power using the Wilcoxon signed rank test compared to the t-test if all the conditions are indeed satisfied, however, the difference is generally very small. --- # Two-sample problems .smallest[In practice, we often encounter problems that compare two samples. For example,] 1. .smallest[A scientist is interested in comparing the vaccine efficacy of the Pfizer-BioNTech and the Moderna vaccine. 🧬🦠] 2. .smallest[A bank wants to know which of two proposed plans will most increase the use of its credit cards. 💸💳] 3. .smallest[A psychologist wants to compare male and female college students' impression on a selected webpage. 📱💻] - .smallest[The goal of inference is to .purple[compare the means of the response variable in two groups.]] - .smallest[Two .hi-turquoise[independent] random samples are separately selected from two .hi-turquoise[distinct] populations, and therefore, they can, for example, be of different sample sizes.] - .smallest[We use .hi-pink[the two-sample t-test] when the population variances are unknown .smallest[👋].] .footnote[.smallest[👋 Similar to the one-sample t-test, we use the two-sample Z-test when both population variances are known, which is very uncommon in practice.]] --- # The two-sample t-test: Example .panelset[ .panel[.panel-name[Problem] .smallest[There is emerging evidence of a relationship between timing of feeding and weight regulation. More precisely, it is claimed that .pink[people who have their main meal early tend to lose more weight compared to people who eat late]. We collect weight loss (in kg) data from 12 individuals, where 7 of them are early eaters and 5 of them are late eaters. We want to test if such claim is valid based on the observed sample: $$\small \text{Early eaters:} \;\; 6.3 \;\; 15.1 \;\; 9.4 \;\; 16.8 \;\; 10.2 \;\; 8.2 \;\; 12.7 \quad \text{Late eaters:} \;\; 7.8 \;\; 0.2 \;\; 1.5 \;\; 11.5 \;\; 4.6 $$ ] 1. .smallest[.purple[Define hypotheses:]] `\(\small H_0: \mu_{\text{early}} = \mu_{\text{late}}\)` .smallest[and] `\(\small H_a: \mu_{\text{early}} \color{#e64173}{>} \mu_{\text{late}}\)`.smallest[.] 2. .smallest[.purple[Define]] `\(\small \color{#6A5ACD}{\alpha}\)`.smallest[: We consider] `\(\small \alpha = 5\%\)`.smallest[.] 3. .smallest[.purple[Compute p-value:] p-value =] `\(\small 2.122\%\)` .smallest[(see R code tab for details).] 4. .smallest[.purple[Conclusion:] We have p-value <] `\(\small \alpha\)` .smallest[so we can reject the null hypothesis at the significance level of 5% and conclude that the early eaters tend to lose more weight compared to late eaters.] ] .panel[.panel-name[Boxplot Code] ```r early_eaters = c(6.3, 15.1, 9.4, 16.8, 10.2, 8.2, 12.7) late_eaters = c(7.8, 0.2, 1.5, 11.5, 4.6) boxplot(early_eaters, late_eaters, names = c("Early", "Late"), ylab = "Weight loss (kg)", col = c("#F8766D", "#00BFC4")) points(1:2, c(mean(early_eaters), mean(late_eaters)), pch = 16, col = 1) ``` ] .panel[.panel-name[Boxplot]  ] .panel[.panel-name[`R` Code ] ```r early_eaters = c(6.3, 15.1, 9.4, 16.8, 10.2, 8.2, 12.7) late_eaters = c(7.8, 0.2, 1.5, 11.5, 4.6) t.test(x = early_eaters, y = late_eaters, alternative = "greater") ``` ``` #> #> Welch Two Sample t-test #> #> data: early_eaters and late_eaters #> t = 2.4351, df = 7.5918, p-value = 0.02122 #> alternative hypothesis: true difference in means is greater than 0 #> 95 percent confidence interval: #> 1.414346 Inf #> sample estimates: #> mean of x mean of y #> 11.24286 5.12000 ``` ] ] --- # Wilcoxon rank sum test - .smallest[Similar to the Wilcoxon signed rank test as a robust .smallest[👋] alternative to the one-sample t-test, we can use the .hi-pink[Wilcoxon rank sum test] (also called the Mann-Whitney test) when the normality condition is not satisfied to use two-sample t-test.] - .smallest[The Wilcoxon rank sum test depends on the .purple[Wilcoxon rank sum statistic], which is the .purple[sum of the ranks of one sample]. As it only uses the ranks of the observations, it is a .turquoise[nonparametric test].] - .smallest[In the simplest form, the Wilcoxon rank sum test states the hypotheses in terms of .pink[population median]. However, more precisely, it actually tests whether the two distributions are the same, i.e. 😵] `\begin{align} & \small H_0: \text{The two distributions are the same.} \\ & \small H_a: \text{One distribution has values that are systematically larger (or smaller). } \end{align}` .footnote[.smallest[👋 Informally, a robust method is such that it is not overly affected by outliers. The usual one-sample and two-sample t-tests are somehow robust in the sense that their results of inference are not very sensitive to moderate lack of normality when the samples are sufficiently large. However, they may still fail when the population distribution shows strong skewness, especially when we have only a few observations.]] --- # Revisit the weight loss example .panelset[ .panel[.panel-name[Problem] .smallest[Let's recall the previous weight loss example. We want to test if .pink[early eaters tend to lose more weight than late eaters]. The sample is as follows: $$\small \text{Early eaters:} \;\; 6.3 \;\; 15.1 \;\; 9.4 \;\; 16.8 \;\; 10.2 \;\; 8.2 \;\; 12.7 \quad \text{Late eaters:} \;\; 7.8 \;\; 0.2 \;\; 1.5 \;\; 11.5 \;\; 4.6 $$ ] 1. .smallest[.purple[Define hypotheses:]] `\(\small H_0: \text{median}_{\text{early}} = \text{median}_{\text{late}}\)` .smallest[and] `\(\small H_a: \text{median}_{\text{early}} \color{#e64173}{>} \text{median}_{\text{late}}\)`.smallest[.] 2. .smallest[.purple[Define]] `\(\small \color{#6A5ACD}{\alpha}\)`.smallest[: We consider] `\(\small \alpha = 5\%\)`.smallest[.] 3. .smallest[.purple[Compute p-value:] p-value =] `\(\small 2.399\%\)` .smallest[(see R code tab for details).] 4. .smallest[.purple[Conclusion:] We have p-value <] `\(\small \alpha\)` .smallest[so we can reject the null hypothesis at the significance level of 5% and conclude that the early eaters tend to lose more weight compared to late eaters.] ] .panel[.panel-name[Boxplot Code] ```r early_eaters = c(6.3, 15.1, 9.4, 16.8, 10.2, 8.2, 12.7) late_eaters = c(7.8, 0.2, 1.5, 11.5, 4.6) boxplot(early_eaters, late_eaters, names = c("Early", "Late"), ylab = "Weight loss (kg)", col = c("#F8766D", "#00BFC4")) points(1:2, c(mean(early_eaters), mean(late_eaters)), pch = 16, col = 1) ``` ] .panel[.panel-name[Boxplot]  ] .panel[.panel-name[`R` Code ] ```r early_eaters = c(6.3, 15.1, 9.4, 16.8, 10.2, 8.2, 12.7) late_eaters = c(7.8, 0.2, 1.5, 11.5, 4.6) wilcox.test(x = early_eaters, y = late_eaters, alternative = "greater") ``` ``` #> #> Wilcoxon rank sum test #> #> data: early_eaters and late_eaters #> W = 30, p-value = 0.02399 #> alternative hypothesis: true location shift is greater than 0 ``` ] ] --- # Exercise An airline company is planning to purchase some new wide-body airliners and narrows down its candidates to Boeing 747 and Lockheed L-1011 TriStar. The company wants to know .pink[if Boeing 747 has longer lifespans than Lockheed L-1011 TriStar] based on the following available samples (in years): `\begin{align} & \text{Boeing} \;\; 30.2 \;\; 27.9 \;\; 29.4 \;\; 22.5 \;\; 26.1 \;\; 22.8 \;\; 27.7 \;\; 27.2 \;\; 31.2 \;\; 29.7 \\ & \text{Lockheed} \;\; 22.4 \;\; 20.2 \;\; 22.9 \;\; 25.1 \;\; 21.8 \;\; 24.2 \;\; 17.1 \;\; 20.5 \;\; 25.8 \;\; 22.6 \\ \end{align}` To import the data, please use the following code: ```r boeing = c(30.2, 27.9, 29.4, 22.5, 26.1, 22.8, 27.7, 27.2, 31.2, 29.7) lockheed = c(22.4, 20.2, 22.9, 25.1, 21.8, 24.2, 17.1, 20.5, 25.8, 22.6) ``` Perform a test you judge appropriate. What conclusion can you draw? 🧐 --- # Solution: the two-sample t-test .panelset[ .panel[.panel-name[Boxplot Code] ```r boeing = c(30.2, 27.9, 29.4, 22.5, 26.1, 22.8, 27.7, 27.2, 31.2, 29.7) lockheed = c(22.4, 20.2, 22.9, 25.1, 21.8, 24.2, 17.1, 20.5, 25.8, 22.6) boxplot(boeing, lockheed, col = c("#F8766D", "#00BFC4"), names = c("Boeing", "Lockheed"), ylab = "Lifespans (year)") points(1:2, c(mean(boeing), mean(lockheed)), pch = 16, col = 1) ``` ] .panel[.panel-name[Boxplot]  ] .panel[.panel-name[Solution] 1. .smallest[.purple[Define hypotheses:]] `\(\small H_0: \mu_{\text{B}} = \mu_{\text{L}}\)` .smallest[and] `\(\small H_a: \mu_{\text{B}} \color{#e64173}{>} \mu_{\text{L}}\)`.smallest[.] 2. .smallest[.purple[Define]] `\(\small \color{#6A5ACD}{\alpha}\)`.smallest[: We consider] `\(\small \color{#e64173}{\alpha = 1\%}\)`.smallest[.] 3. .smallest[.purple[Compute p-value:] p-value =] `\(\small 0.02739\%\)` .smallest[(see R code tab for details).] 4. .smallest[.purple[Conclusion:] We have p-value <] `\(\small \alpha\)` .smallest[so we can reject the null hypothesis at the significance level of 1% and conclude that Boeing 747 has longer lifespans than Lockheed L-1011 TriStar.] ] .panel[.panel-name[`R` Code ] ```r boeing = c(30.2, 27.9, 29.4, 22.5, 26.1, 22.8, 27.7, 27.2, 31.2, 29.7) lockheed = c(22.4, 20.2, 22.9, 25.1, 21.8, 24.2, 17.1, 20.5, 25.8, 22.6) t.test(x = boeing, y = lockheed, alternative = "greater") ``` ``` #> #> Welch Two Sample t-test #> #> data: boeing and lockheed #> t = 4.2098, df = 17.641, p-value = 0.0002739 #> alternative hypothesis: true difference in means is greater than 0 #> 95 percent confidence interval: #> 3.061581 Inf #> sample estimates: #> mean of x mean of y #> 27.47 22.26 ``` ] ] --- # Solution: Wilcoxon rank sum test .panelset[ .panel[.panel-name[Boxplot Code] ```r boeing = c(30.2, 27.9, 29.4, 22.5, 26.1, 22.8, 27.7, 27.2, 31.2, 29.7) lockheed = c(22.4, 20.2, 22.9, 25.1, 21.8, 24.2, 17.1, 20.5, 25.8, 22.6) boxplot(boeing, lockheed, col = c("#F8766D", "#00BFC4"), names = c("Boeing", "Lockheed"), ylab = "Lifespans (year)") points(1:2, c(mean(boeing), mean(lockheed)), pch = 16, col = 1) ``` ] .panel[.panel-name[Boxplot]  ] .panel[.panel-name[Solution] 1. .smallest[.purple[Define hypotheses:]] `\(\small H_0: \text{median}_{\text{B}} = \text{median}_{\text{L}}\)` .smallest[and] `\(\small H_a: \text{median}_{\text{B}} \color{#e64173}{>} \text{median}_{\text{L}}\)`.smallest[.] 2. .smallest[.purple[Define]] `\(\small \color{#6A5ACD}{\alpha}\)`.smallest[: We consider] `\(\small \color{#e64173}{\alpha = 1\%}\)` .smallest[(see R code tab for details).] 3. .smallest[.purple[Compute p-value:] p-value =] `\(\small 0.0525\%\)`.smallest[.] 4. .smallest[.purple[Conclusion:] We have p-value <] `\(\small \alpha\)` .smallest[so we can reject the null hypothesis at the significance level of 1% and conclude that Boeing 747 has longer lifespans than Lockheed L-1011 TriStar.] ] .panel[.panel-name[`R` Code ] ```r boeing = c(30.2, 27.9, 29.4, 22.5, 26.1, 22.8, 27.7, 27.2, 31.2, 29.7) lockheed = c(22.4, 20.2, 22.9, 25.1, 21.8, 24.2, 17.1, 20.5, 25.8, 22.6) wilcox.test(x = boeing, y = lockheed, alternative = "greater") ``` ``` #> #> Wilcoxon rank sum test #> #> data: boeing and lockheed #> W = 91, p-value = 0.000525 #> alternative hypothesis: true location shift is greater than 0 ``` ] ] --- # Why are we only comparing the means? - .smaller[So far we have only discussed tests for comparison of means. However, in fact, .purple[the comparison of other features of the data such as the variance can be of great interest as well.] A good example of this is the very controversial .hi-pink["variability hypothesis"]. ] - .smaller[The variability hypothesis has often been discussed in relation to cognitive ability, where it has been observed that human males are more likely than females to have either very high or very low intelligence. The .pink[sex difference in the variability of intelligence] has been discussed since at least Charles Darwin. The variability hypothesis has continued to spur controversy within academic circles. ] .footnote[.smallest[👋 If you are interested in knowing more, check out [here](https://en.wikipedia.org/wiki/Variability_hypothesis#:~:text=The%20variability%20hypothesis%2C%20also%20known,in%20traits%20than%20females%20do.&text=Sex%2Ddifferences%20in%20variability%20are,physical%2C%20psychological%20and%20genetic%20ones.)!]] --- # How about more than 2 samples? In practice, we often even encounter situations where we need to .pink[compare the means of more than 2 groups]. For example, 1. A scientist wants to compare the vaccine efficacy of the Pfizer-BioNTech, the Moderna, and the Sputnik V vaccines. 2. A nutritionist wants to compare the weight loss efficacy of several diets. 3. A restaurant wants to compare the consumptions among 3 different kinds of pizzas. 🍕🍴 --- # Jelly beans example .purple[Are jelly beans causing acne? Maybe... but why only green ones?] 🤨  .tiny[Source: [xkcd](https://xkcd.com/882/)] --- # Are jelly beans causing acne?
  .tiny[Source: [xkcd](https://xkcd.com/882/)] --- # Maybe a specific color?
  .tiny[Source: [xkcd](https://xkcd.com/882/)] --- # Maybe a specific color?
  .tiny[Source: [xkcd](https://xkcd.com/882/)] --- # And finally...  .tiny[Source: [xkcd](https://xkcd.com/882/)] 👋 .smallest[If you want to know more about this comics have a look [here](https://www.explainxkcd.com/wiki/index.php/882:_Significant).] --- # ☠ Multiple testing can be dangerous! - .smaller[Remember that a p-value is .hi-pink[random] as its value depends on the data.] - .smaller[Hence, by chance, it might happen that while the null hypothesis cannot be rejected (supposing it is true), that the p-value is smaller than the set threshold. With the latter chosen as 5%, then, on average, the (sample) p-value is below 5% .purple[once in twenty times!]] - .smaller[If multiple hypotheses are tested, the chance of observing a rare event increases, and therefore, the chance to incorrectly reject a null hypothesis (i.e., making a Type I error) increases.] - .smaller[Hence .hi-pink[performing multiple tests, with the same or different data, is dangerous ⚠] (but very common! 😟) as it automatically leads to .pink[significant results, when actually there are none!]] --- # ☠ Multiple testing can be dangerous!  .footnote[.smallest[👋] Read the original article: "*This is why you shouldn't believe that exciting new medical study*" [here](https://www.vox.com/2015/3/23/8264355/research-study-hype).] --- # Methods to remedy such problem - .smallest[The .hi-pink[Bonferroni correction] is one of the oldest methods proposed to counteract the problem of multiple testing (i.e. potential increased chance to make Type I errors). ] - .smallest[.purple[It tests each individual hypothesis at a reduced significance level.] For example, if a trial is testing 20 hypotheses with a desired] `\(\small \alpha = 5\%\)`.smallest[, then the Bonferroni correction would test each individual hypothesis at] `\(\small \alpha^* = \alpha/20 = 5\% / 20 = 0.25\%\)`.smallest[. Therefore, .purple[it controls the familywise error rate at]] `\(\small \color{#6A5ACD}{\leq \alpha}\)`. - .smallest[The Bonferroni correction can be .turquoise[conservative] if there are a large number of tests, as it comes at the cost of reducing the power of the tests.] - .smallest[Another more recent and less conservative method is the .hi-pink[False Discovery Rate (FDR)]. It provides less stringent control of Type I errors compared to the Bonferroni correction. Thus, the FDR has greater power, yet at the cost of increased numbers of Type I errors.] --- # The one-way ANOVA - .smallest[To compare several means of different populations, a standard approach is to start our analysis by using the .hi-pink[(one-way) ANalysis Of VAriance (ANOVA)]. More precisely, we proceed our analysis with the following steps:] - .smallest[.purple[Step 1:] We first perform the ANOVA, where the null hypothesis states that all the means are the same. If we cannot reject the null hypothesis, we stop our analysis here. Otherwise, we move on to Step 2.] - .smallest[.purple[Step 2:] We compare the groups mutually with two-sample t-tests in order to verify our hypothesis.] - .smallest[For example, suppose we have 3 groups and our hypothesis is that the mean of Group 1 is larger than the mean of Group 2, which is larger than the mean of Group 3. We start with an ANOVA and test if all the means are equal. If we reject the null hypothesis, then we perform the two-sample t tests of Group 1 vs Group 2, Group 1 vs Group 3, and Group 2 vs Group 3, which in total are 3 tests.] - .smallest[.pink[Naturally, we should adjust the significance level for multiple testing in Step 2.]] --- # The one-way ANOVA .smallest[We are interested in comparing means, but why is it an analysis of .purple[variance]?] .pull-left[  ] .pull-right[  ] --- # The one-way ANOVA .smallest[The one-way ANOVA makes similar assumptions as the two-sample t-test:] - `\(\small J\)` .smallest[.hi-turquoise[independent] random samples are separately selected from each of these .hi-turquoise[distinct] populations. They can be of different sample sizes.] - .smallest[The] `\(\small j^{th}\)` .smallest[population has a .pink[normal distribution] with unknown mean] `\(\small \mu_j\)`. - .smallest[.purple[All the populations have the same variance], whose value is unknown.] .smallest[Then the hypotheses for the one-way ANOVA can be specified as follows:] `\begin{align} & \small H_0: \mu_1 = \mu_2 = \ldots = \mu_J \\ & \small H_a: H_0 \text{ is false} \end{align}` .smallest[⚠ Notice that the alternative says that the null hypothesis is false, which can be true because all the means are different or simply because one of them differs from the rest. So this is a more complex situation than comparing just two populations. .pink[If we reject the null hypothesis, we need to perform some further analysis to draw conclusions about which population means differ from which others and by how much.]] --- # The one-way ANOVA: Example .panelset[ .panel[.panel-name[Prob] .smallest[Lamb's-quarter is a common weed that interferes with the growth of corn. A researcher collected 3 samples of corn yields (bushels per acre) from 3 plots, each of which are weeded such that there are 1, 3, 9 units of weed per squared meter. She wants to show that more weed lead to less corn yields with the following data:] `\begin{align} & \small \text{Group 1 (1 weed/squared meter): } 163.7 \;\; 162.6 \;\; 155.4 \;\; 158.8 \;\; 156.1 \\ & \small \text{Group 2 (3 weed/squared meter): } 154 \;\; 152.7 \;\; 160.9 \;\; 157.5 \;\; 154.4 \\ & \small \text{Group 3 (9 weed/squared meter): } 149.9 \;\; 148.4 \;\; 139.5 \;\; 149.1 \;\; 145.2 \end{align}` ] .panel[.panel-name[Boxplot] <img src="lecture2_files/figure-html/unnamed-chunk-36-1.svg" width="80%" style="display: block; margin: auto;" /> ] .panel[.panel-name[`R` Code] ```r corn1 = c(163.7, 162.6, 155.4, 158.8, 156.1) corn2 = c(154, 152.7, 160.9, 157.5, 154.4) corn3 = c(149.9, 148.4, 139.5, 149.1, 145.2) boxplot(corn1, corn2, corn3, names = c("1 weed", "3 weed", "9 weed"), ylab = "Corn yields (bushels per acre)", col = c("#F8766D", "#00BA38", "#619CFF")) points(1:3, c(mean(corn1), mean(corn2), mean(corn3)), pch = 16, col = 1) ``` ] .panel[.panel-name[ANOVA] 1. .smallest[.purple[Define hypotheses:]] `\begin{align} &\small H_0: \mu_1 = \mu_2 = \mu_3 \\ &\small H_a: H_0 \text{ is false} \end{align}` 2. .smallest[.purple[Define]] `\(\small \color{#6A5ACD}{\alpha}\)`.smallest[: We consider] `\(\small \alpha = 5\%\)`.smallest[.] 3. .smallest[.purple[Compute p-value]: p-value = ] `\(\small 0.0464\%\)` .smallest[(see R code tab for details).] 4. .smallest[.purple[Conclusion:] We have p-value <] `\(\small \alpha\)` .smallest[so we can reject the null hypothesis at the significance level of 5% and conclude that not all the means of corn yields are the same among these 3 groups.] ] .panel[.panel-name[`R` Code] ```r corn1 = c(163.7, 162.6, 155.4, 158.8, 156.1) corn2 = c(154, 152.7, 160.9, 157.5, 154.4) corn3 = c(149.9, 148.4, 139.5, 149.1, 145.2) corns = c(corn1, corn2, corn3) groups = c(rep("Group 1", 5), rep("Group 2", 5), rep("Group 3", 5)) res_aov = aov(corns ~ groups) summary(res_aov) ``` ``` #> Df Sum Sq Mean Sq F value Pr(>F) #> groups 2 446.6 223.31 15.56 0.000464 *** #> Residuals 12 172.2 14.35 #> --- #> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ``` ] .panel[.panel-name[Comp] ```r t.test(corn1, corn2, alternative = "greater") t.test(corn1, corn3, alternative = "greater") t.test(corn2, corn3, alternative = "greater") ``` These results suggest, at the adjusted significance level of `\(5\%/3 \approx 1.67\%\)`, that more weed lead to less corn yields. ] ] --- # Kruskal-Wallis test - .smallest[The .hi-pink[Kruskal-Wallis test] is a test that can replace the one-way ANOVA. The assumption about data production (i.e. independent random samples from each population) remains important, but .pink[we can relax the normality and the same variance assumptions.] Indeed, we only need to assume that the response has a continuous distribution in each population.] - .smallest[Just like the Wilcoxon tests for one and two samples, .purple[the Kruskal-Wallis test is particularly useful when there are outliers and/or when the normality condition cannot be satisfied.]] - .smallest[The idea of the Kruskal-Wallis test is to rank all the responses from all groups together and then apply the one-way ANOVA to the .turquoise[ranks] rather than to the original observations. Therefore, it is a .turquoise[nonparametric test].] - .smallest[In the simplest form, the Kruskal-Wallis test states the hypotheses in terms of .pink[population median]. That is,] `\begin{align} & \small H_0: \text{all populations have the same median} \\ & \small H_a: H_0 \text{ is false} \end{align}` --- # Revisit the corn yields example .panelset[ .panel[.panel-name[Prob] .smallest[Let's recall the previous corn yields example. The researcher wants to test if more weed lead to less corn yields. Recall the data as follows:] `\begin{align} & \small \text{Group 1 (1 weed/squared meter): } 163.7 \;\; 162.6 \;\; 155.4 \;\; 158.8 \;\; 156.1 \\ & \small \text{Group 2 (3 weeds/squared meter): } 154 \;\; 152.7 \;\; 160.9 \;\; 157.5 \;\; 154.4 \\ & \small \text{Group 3 (9 weeds/squared meter): } 149.9 \;\; 148.4 \;\; 139.5 \;\; 149.1 \;\; 145.2 \end{align}` ] .panel[.panel-name[Boxplot] <img src="lecture2_files/figure-html/unnamed-chunk-40-1.svg" width="80%" style="display: block; margin: auto;" /> ] .panel[.panel-name[`R` Code] ```r corn1 = c(163.7, 162.6, 155.4, 158.8, 156.1) corn2 = c(154, 152.7, 160.9, 157.5, 154.4) corn3 = c(149.9, 148.4, 139.5, 149.1, 145.2) boxplot(corn1, corn2, corn3, names = c("Group 1", "Group 2", "Group 3"), ylab = "Corn yields (bushels per acre)", col = c("#F8766D", "#00BA38", "#619CFF")) points(1:3, c(mean(corn1), mean(corn2), mean(corn3)), pch = 16, col = 1) ``` ] .panel[.panel-name[Kruskal-Wallis] 1. .smallest[.purple[Define hypotheses:]] `\begin{align} &\small H_0: \text{median}_1 = \text{median}_2 = \text{median}_3 \\ &\small H_a: H_0 \text{ is false} \end{align}` 2. .smallest[.purple[Define]] `\(\small \color{#6A5ACD}{\alpha}\)`.smallest[: We consider] `\(\small \alpha = 5\%\)`.smallest[.] 3. .smallest[.purple[Compute p-value]: p-value = ] `\(\small 0.5248\%\)` .smallest[(see R code tab for details).] 4. .smallest[.purple[Conclusion:] We have p-value <] `\(\small \alpha\)` .smallest[so we can reject the null hypothesis at the significance level of 5% and conclude that not all the means of corn yields are the same among these 3 groups that are weeded differently.] ] .panel[.panel-name[`R` Code] ```r corn1 = c(163.7, 162.6, 155.4, 158.8, 156.1) corn2 = c(154, 152.7, 160.9, 157.5, 154.4) corn3 = c(149.9, 148.4, 139.5, 149.1, 145.2) corns = c(corn1, corn2, corn3) groups = c(rep("Group 1", 5), rep("Group 2", 5), rep("Group 3", 5)) kruskal.test(corns ~ groups) ``` ``` #> #> Kruskal-Wallis rank sum test #> #> data: corns by groups #> Kruskal-Wallis chi-squared = 10.5, df = 2, p-value = 0.005248 ``` ] .panel[.panel-name[Comp] ```r wilcox.test(corn1, corn2, alternative = "greater") wilcox.test(corn1, corn3, alternative = "greater") wilcox.test(corn2, corn3, alternative = "greater") ``` These results suggest, at the adjusted significance level of `\(5\%/3 \approx 1.67\%\)`, that more weed lead to less corn yields. ] ]


Keiichi
Morisato is a fairly average college studen attending the Nekomi Institute of
Technology. He stays in the men's dorm and is left to take messages for the
other members of the dormitory. As he waits for his friends to return, he gets
hungry and orders for food to be delivered to his place. However, the number he
dials is not the number of a Chinese take-out restaurant. He ends up dialing the
number to the Goddess help line and help is being sent to him immediately.
Through a mirror on the wall pops a young woman dressed in flowing robes.
Keiichi is startled and scared, but curious at the same time. Believing it to be
a prank put on by Tamiya and Otaki, he goes along with her claims to be a
goddess. However, as she continues her tales, he begins to believe her, and
being down on his luck, makes an incredible wish. For her to be with him
forever.
Keiichi's
wish is granted and the goddess Belldandy is to stay by his side forever.
However, women aren't allowed in the dorms and Tamiya enforces this strictly.
 This
is the work of the Ultimate Force, which will keep Belldandy and Keiichi
together at all costs. Eventually, the two find an old temple and Belldandy
repairs it.
However, Keiichi's possession of a house leads to other things. His sister
Megumi moves in for a short while and eventually both of Belldandy's sisters
make their way down from the heaven. Urd who makes it her quest to get Belldandy
and Keiichi closer together and Skuld who wants nothing more than to be with her
sister. Add in the hectic antics of the NIT Motor Cycle Club led by Tamiya and
Otaki and it's a regular riot at the Morisato residence. The pair is also under
constant pressure from Sayoko who
wants Keiichi for herself and Aoshima who wants Belldandy for himself.  Finally,
there's Peorth another first class goddess from a different office. Keiichi
(again) dials the wrong number and Peorth arrives at his residence. Keiichi
refuses to take a wish from Peorth and she takes it quite personally creating a
grudge against Belldandy. She also wants to be closer to Keiichi.
But the love that has formed between Belldandy and Keiichi will remain strong
forever. Not even the touch of an old flame, another Goddess, a meddling younger
sister, threats from other students or even the decree of God himself have been
able to tear the two apart. It was a promise made even years before they met at
NIT, and it will always be a promise kept.


Nice To Know
IT
Data Science & Engineering
Machine Learning
Tensorflow
open source library for machine learning
https://www.tensorflow.org
https://github.com/tensorflow/tensorflow


VTA Installation Guide¶
We present three installation guides, each extending on the previous one: 1. [Simulator installation](#vta-simulator-installation) 2. [Hardware test setup](#vta-pynq-based-test-setup) 3. [FPGA toolchain installation](#vta-fpga-toolchain-installation)
## VTA Simulator Installation
You need [TVM installed](https://docs.tvm.ai/install/index.html) on your machine. For a quick and easy start, use the pre-built [TVM Docker image](https://docs.tvm.ai/install/docker.html).
The VTA simulator library is built by default with TVM. Add the VTA library to your python path to run the VTA examples.
`bash
export PYTHONPATH=/path/to/vta/python:${PYTHONPATH}
`
### Testing your VTA Simulation Setup
To ensure that you’ve properly installed the VTA python package, run the following 2D convolution testbench.
`bash
python <tvm root>/vta/tests/python/integration/test_benchmark_topi_conv2d.py
`
> Note: You’ll notice that for every convolution layer, the throughput gets reported in GOPS. These numbers are actually the computational throughput that the simulator achieves, by evaluating the convolutions in software.
You are invited to try out our [VTA programming tutorials](https://docs.tvm.ai/vta/tutorials/index.html).
### Advanced Configuration (optional)
VTA is a generic configurable deep learning accelerator. The configuration is specified by vta_config.json under the TVM root folder. This file provides an architectural specification of the VTA accelerator to parameterize the TVM compiler stack and the VTA hardware stack.
The VTA configuration file also specifies the TVM compiler target. When TARGET is set to sim, all TVM workloads execute on the VTA simulator. You can modify the content of the configuration file to rebuild VTA to a different parameterization. To do so,
`bash
cd <tvm root>
cp vta/config/vta_config.json vta_config.json
# edit vta_config.json
make vta
`
## VTA Pynq-Based Test Setup
This second guide extends the VTA Simulator Installation guide above to run FPGA hardware tests of the complete TVM and VTA software-hardware stack. In terms of hardware components you’ll need: * The [Pynq](http://www.pynq.io/) FPGA development board which can be acquired for $200, or $150 for academics from [Digilent](https://store.digilentinc.com/pynq-z1-python-productivity-for-zynq/). * An Ethernet-to-USB adapter to connect the Pynq board to your development machine. * An 8+GB micro SD card. * An AC to DC 12V 3A power adapter.
This guide covers the following themes: 1. Pynq board setup instructions. 2. Pynq-side RPC server build and deployment. 3. Revisiting the test examples from the VTA Simulator Installation guide, this time executing on the Pynq board.
### Pynq Board Setup
Setup your Pynq board based on the [Pynq board getting started tutorial](http://pynq.readthedocs.io/en/latest/getting_started.html). You should follow the instructions up to and including the Turning On the PYNQ-Z1 step (no need to pursue the tutorial beyond this point). * Make sure that you’ve downloaded the latest Pynq image, [PYNQ-Z1 v2.1](http://pynq-testing.readthedocs.io/en/image_v2.2/getting_started/pynq_image.html) (released 21 Feb 2018), and have imaged your SD card with it (we recommend the free [Etcher](https://etcher.io/) program). * For this test setup, follow the [“Connect to a Computer”](http://pynq.readthedocs.io/en/latest/getting_started.html#connect-to-a-computer) Ethernet setup instructions. To be able to talk to the board, make sure to [assign your computer a static IP address](http://pynq.readthedocs.io/en/latest/appendix.html#assign-your-computer-a-static-ip)
Once the board is powered on and connected to your development machine, try connecting to it to make sure you’ve properly set up your Pynq board:
`bash
# To connect to the Pynq board use the [username, password] combo: [xilinx, xilinx]
ssh<EMAIL_ADDRESS>`
### Pynq-Side RPC Server Build & Deployment
Because the direct board-to-computer connection prevents the board from directly accessing the internet, we’ll need to mount the Pynq’s file system to your development machine’s file system with [sshfs](https://www.digitalocean.com/community/tutorials/how-to-use-sshfs-to-mount-remote-file-systems-over-ssh). Next we directly clone the TVM repository into the sshfs mountpoint on your development machine.
`bash
# On the Host-side
mkdir <mountpoint>
sshfs xilinx@<IP_ADDRESS>:/home/xilinx <mountpoint>
cd <mountpoint>
git clone --recursive https://github.com/dmlc/tvm
# When finished, you can leave the moutpoint and unmount the directory
cd ~
sudo umount <mountpoint>
`
Now that we’ve cloned the VTA repository in the Pynq’s file system, we can ssh into it and launch the build of the TVM-based RPC server. The build process should take roughly 5 minutes.
`bash
ssh<EMAIL_ADDRESS># Build TVM runtime library (takes 5 mins)
cd /home/xilinx/tvm
mkdir build
cp cmake/config.cmake build/.
# Copy pynq specific configuration
cp vta/config/pynq_sample.json build/vta_config.json
cd build
cmake ..
make runtime vta -j2
# Build VTA RPC server (takes 1 min)
cd ..
sudo ./apps/pynq_rpc/start_rpc_server.sh # pw is 'xilinx'
`
You should see the following being displayed when starting the RPC server. In order to run the next examples, you’ll need to leave the RPC server running in an ssh session.
`
INFO:root:RPCServer: bind to <IP_ADDRESS>:9091
`
Tips regarding the Pynq RPC Server: * The RPC server should be listening on port 9091. If not, an earlier process might have terminated unexpectedly and it’s recommended in this case to just reboot the Pynq, and re-run the RPC server. * To kill the RPC server, just send the Ctrl + c command. You can re-run it with sudo ./apps/pynq_rpc/start_rpc_server.sh. * If unresponsive, the board can be rebooted by power-cycling it with the physical power switch.
### Testing your Pynq-based Hardware Setup
Before running the examples on your development machine, you’ll need to configure your host environment as follows:
`bash
# On the Host-side
export VTA_PYNQ_RPC_HOST=<IP_ADDRESS>
export VTA_PYNQ_RPC_PORT=9091
`
In addition, you’ll need to edit the vta_config.json file on the host to indicate that we are targeting the Pynq platform, by setting the TARGET field to “pynq”. Alternatively, you can copy the default vta/config/pynq_sample.json into the TVM root as vta_config.json. > Note: in contrast to our simulation setup, there are no libraries to compile on the host side since the host offloads all of the computation to the Pynq board.
`bash
# On the Host-side
cd <tvm root>
cp vta/config/pynq_sample.json vta_config.json
`
This time again, we will run the 2D convolution testbench. Beforehand, we need to program the Pynq board FPGA with a VTA bitstream, and build the VTA runtime via RPC. The following test_program_rpc.py script will perform two operations: * FPGA programming, by downloading a pre-compiled bitstream from a [VTA bitstream repository](https://github.com/uwsaml/vta-distro) that matches the default vta_config.json configuration set by the host, and sending it over to the Pynq via RPC to program the Pynq’s FPGA. * Runtime building on the Pynq, which needs to be run every time the vta_config.json configuration is modified. This ensures that the VTA software runtime that generates the accelerator’s executable via just-in-time (JIT) compilation matches the specifications of the VTA design that is programmed on the FPGA. The build process takes about 30 seconds to complete so be patient!
`bash
# On the Host-side
python <tvm root>/vta/tests/python/pynq/test_program_rpc.py
`
> Tip: You can track progress of the FPGA programming and the runtime rebuilding steps by looking at the RPC server’s logging messages in your Pynq ssh session.
We are now ready to run the 2D convolution testbench in hardware.
`bash
# On the Host-side
python <tvm root>/vta/tests/python/integration/test_benchmark_topi_conv2d.py
`
The performance metrics measured on the Pynq board will be reported for each convolutional layer.
You can also try out our [VTA programming tutorials](https://docs.tvm.ai/vta/tutorials/index.html).
## VTA FPGA Toolchain Installation
This third and last guide allows users to generate custom VTA bitstreams using free-to-use Xilinx compilation toolchains.
### Xilinx Toolchain Installation
We recommend using Vivado 2018.2 since our scripts have been tested to work on this version of the Xilinx toolchains. Our guide is written for Linux (Ubuntu) installation.
You’ll need to install Xilinx’ FPGA compilation toolchain, [Vivado HL WebPACK 2018.2](https://www.xilinx.com/products/design-tools/vivado.html), which a license-free version of the Vivado HLx toolchain.
#### Obtaining and Launching the Vivado GUI Installer
Go to the [download webpage](https://www.xilinx.com/support/download/index.html/content/xilinx/en/downloadNav/vivado-design-tools/2018-2.html), and download the Linux Self Extracting Web Installer for Vivado HLx 2018.2: WebPACK and Editions.
You’ll have to sign in with a Xilinx account. This requires a Xilinx account creation that will take 2 minutes.
Complete the Name and Address Verification by clicking “Next”, and you will get the opportunity to download a binary file, called Xilinx_Vivado_SDK_Web_2018.2_0614_1954_Lin64.bin.
4. Now that the file is downloaded, go to your Downloads directory, and change the file permissions so it can be executed:
`bash
chmod u+x Xilinx_Vivado_SDK_Web_2018.2_0614_1954_Lin64.bin
`
5. Now you can execute the binary:
`bash
./Xilinx_Vivado_SDK_Web_2018.2_0614_1954_Lin64.bin
`
#### Xilinx Vivado GUI Installer Steps
At this point you’ve launched the Vivado 2017.1 Installer GUI program.
Click “Next” on the Welcome screen.
On the Select Install Type screen, enter your Xilinx user credentials under the “User Authentication” box and select the “Download and Install Now” option before clicking “Next” .
On the Accept License Agreements screen, accept all terms before clicking “Next”.
On the Select Edition to Install screen, select the “Vivado HL WebPACK” before clicking “Next” .
Under the Vivado HL WebPACK screen, before hitting “Next”, check the following options (the rest should be unchecked): * Design Tools -> Vivado Design Suite -> Vivado * Devices -> Production Devices -> SoCs -> Zynq-7000 (if you are targeting the Pynq board) * Devices -> Production Devices -> SoCs -> UltraScale+ MPSoC (if you are targeting the Ultra-96 board)
Your total download size should be about 5GB and the amount of Disk Space Required 23GB.
On the Select Destination Directory screen, set the installation directory before clicking “Next”. It might highlight some paths as red - that’s because the installer doesn’t have the permission to write to the directory. In that case select a path that doesn’t require special write permissions (e.g. your home directory).
On the Installation Summary screen, hit “Install”.
An Installation Progress window will pop-up to track progress of the download and the installation.
This process will take about 20-30 minutes depending on your connection speed.
A pop-up window will inform you that the installation completed successfully. Click “OK”.
Finally the Vivado License Manager will launch. Select “Get Free ISE WebPACK, ISE/Vivado IP or PetaLinux License” and click “Connect Now” to complete the license registration process.
#### Environment Setup
The last step is to update your ~/.bashrc with the following lines. This will include all of the Xilinx binary paths so you can launch compilation scripts from the command line.
`bash
# Xilinx Vivado 2018.2 environment
export XILINX_VIVADO=${XILINX_PATH}/Vivado/2018.2
export PATH=${XILINX_VIVADO}/bin:${PATH}
`
### Custom VTA Bitstream Compilation
High-level hardware parameters are listed in the VTA configuration file and can be customized by the user. For this custom VTA bitstream compilation exercise, we’ll change the frequency of our design, so it can be clocked a little faster. * Set the HW_FREQ field to 142. The Pynq board supports 100, 142, 167 and 200MHz clocks. Note that the higher the frequency, the harder it will be to close timing. Increasing the frequency can lead to timing violation and thus faulty hardware execution. * Set the HW_CLK_TARGET to 6. This parameters refers to the target clock period in nano seconds for HLS - a lower clock period leads to more aggressive pipelining to achieve timing closure at higher frequencies. Technically a 142MHz clock would require a 7ns target, but we intentionally lower the clock target to 6ns to more aggressively pipeline our design.
Bitstream generation is driven by a top-level Makefile under <tvm root>/vta/hardware/xilinx/.
If you just want to simulate the VTA design in software emulation to make sure that it is functional, enter:
`bash
cd <tvm root>/vta/hardware/xilinx
make ip MODE=sim
`
If you just want to generate the HLS-based VTA IP cores without launching the entire design place and route, enter:
`bash
make ip
`
You’ll be able to view the HLS synthesis reports under <tvm root>/vta/build/hardware/xilinx/hls/ <configuration>/<block>/solution0/syn/report/<block>_csynth.rpt
> Note: The <configuration> name is a string that summarizes the VTA configuration parameters listed in the vta_config.json. The <block> name refers to the specific module (or HLS function) that compose the high-level VTA pipeline.
Finally to run the full hardware compilation and generate the VTA bitstream, run:
`bash
make
`
This process is lengthy, and can take around up to an hour to complete depending on your machine’s specs. We recommend setting the VTA_HW_COMP_THREADS variable in the Makefile to take full advantage of all the cores on your development machine.
Once the compilation completes, the generated bitstream can be found under <tvm root>/vta/build/hardware/xilinx/vivado/<configuration>/export/vta.bit.
### Use the Custom Bitstream
We can program the new VTA FPGA bitstream by setting the bitstream path of the vta.program_fpga() function in the tutorial examples, or in the test_program_rpc.py script.
`python
vta.program_fpga(remote, bitstream="<tvm root>/vta/build/hardware/xilinx/vivado/<configuration>/export/vta.bit")
`
Instead of downloading a pre-built bitstream from the VTA bitstream repository, TVM will instead use the new bitstream you just generated, which is a VTA design clocked at a higher frequency. Do you observe a noticeable performance increase on the ImageNet classification example?


The finite state machine (FSM) has been one of the most popular models for
describing control-oriented systems, like real-time process controllers.
A directed node-and-arc graph, called a state transition diagram (STD),
can be used to describe an FSM. It represents a system in the form of states
(nodes) and transitions (arcs) between states.
2 Graphical User Interface
The old and outdated visual interface to Ptolemy, called VEM, is not suitable
for drawing the STD for an FSM. A new visual editor* (see
Figure) is under development based on Tycho, a hierarchical syntax manager,
which is part of the Ptolemy project.
2.1 Edit a new STD file
To edit a new STD for an FSM, the Tycho has to be started first. Then either
open a file with the file extension "std", or drag down the "Window" menu and
choose "State transition diagram editor". A message window will pop up to ask
for machine type. Currently there is one type of FSM supported, called mixed
Mealy/Moore machine.
2.2 Edit the Input/Output and Internal Events Names
If there are any input/output and internal events for the FSM, their names
must be specified: Drag down the "Special" menu and choose "I/O Port Names..."
, then enter the names for the input/output. Each name should be separated by
at least one space.
2.2 Draw/Edit a State
To draw a state, either drag down the "Edit" menu and choose "New Node",
or press the hot key "Shift-N". Then a crossbar cursor will appear in the
window. Press and hold (don't release) the left mouse button, and move the
mouse to get a different shape of node. Release the button to finish the
drawing.
To edit a state, first select the state by pressing the left mouse button
on it. Then either drag down the "Edit" menu and choose "Edit Item", or
press the hot key "e".
2.3 Draw/Edit a Transition
To draw a transition, either drag down the "Edit" menu and choose "New Arc",
or press the hot key "Shift-A". Then a crossbar cursor will appear in the
window. Press the left mouse button on the periphery of the starting state.
Move the mouse and press the left mouse button to get more delicate arc.
To finish the drawing, move the mouse on the periphery of the ending state
and press the left mouse button.
To edit a transition, first select the transition by pressing the left mouse
button on it. Then either drag down the "Edit" menu and choose "Edit Item",
or press the hot key "e".
2.4 Delete a State/Transition
Fisrt select the state/transition by pressing the left mouse button on it,
then drag down the "Edit" menu and choose "Delete".
2.5 Move/Reshape a State/Transition
To move a state, press and hold the left mouse button on it, then move the
mouse to the desired position. To reshape a state, first select the state,
and move the cursor to the up/down/left/right periphery of the state to get
an up/down/left/right arrow-shape cursor, then press and hold the left mouse
button and move the mouse to reshape it.
To move/reshape a transition, first select it, then some small rectagles will
appear along the arc. Press and move one of the rectagles to move/reshape the
arc.
2.6 Internal Machines of States
In the FSM, each state may be associated with an internal machine. This
internal machine can be a subsystem of any other Ptolemy domain or another
FSM Galaxy.
To specify the internal machine of a state, when edit a state, give the full
path name and file name of the subsystem. For example, an VEM facet for the
Galaxy of any other Ptolemy domain, or an STD file for the FSM Galaxy.
3 Working in Ptolemy
From the point of view of implementation, a stand-alone FSM domain in Ptolemy
is not very interesting, because in most of applications, in addition to the
control part, they contains many other parts, such as signal processing.
Moreover, there are various Ptolemy domains. With the FSM domain mixed with
them**, we can get a much more powerful FSM model.
3.1 Make an Icon in VEM
Unlike other domains, an FSM galaxy is edited in a previous mentioned STD file
using Tycho instead of VEM. However, to work with other domains, an icon in
VEM is required to represent the corresponding FSM galaxy described in an STD
file. In Ptolemy, the Tycho can be started by pressing key "Y". To make an
icon in VEM for the STD file, first open the file in term of Tycho started in
Ptolemy. Then drag down the "Special" menu and choose "Make Icon...". This
will load the FSM galaxy into the Ptolemy kernel, and make an icon with
appropriate input/output ports in the specified palette in VEM.
The icon looks like a star (blue outline), but it is actually an galaxy.
It may cause confusion, but the reason is to avoid using octool handle.
There are two different ways to make an icon in VEM: ptkSetMkSchemIcon
and ptkSetMkStar. The former one needs an OctFactHandle as one argument
and is used for other "VEM" galaxy. The FSM galaxy uses Tycho as the editor
instead of VEM, so there is no OctFactHandle for an FSM galaxy. Therefore
the latter one, which uses the star (or galaxy) name instead, is more
appropriate for this case.
3.2 Look Inside an FSM Galaxy
Similar to the galaxy icon in other domain, when the key "I" is pressed on
the icon to look inside the galaxy, the corresponding STD file which describes
the FSM galaxy will be automatically invoked in Tycho.
(Note: if the environment variable "PT_DISPLAY" is set to other editor instead
of Tycho, it must be unset for the look-inside to work.)
*The drawing mechanisms of this editor have been mainly developed by Wan-teh
Chang.
**Currently, the FSM domain can work with the SDF domain without problem.
Reference:
[1] Bilung Lee, "Fusing Dataflow with Finite State Machines",
Technical Report, Memorandum No. UCB/ERL M96/20, University of California
at Berkeley, May 1996.


AbstractFactory
AbstractManager
FixedDelayTimer
FixedRateTimer
FooRemoteWorkItem
FooService
FooServiceMBean
FooTimer
FooTimerManager
FooTimerManagerFactory
FooWorkEvent
FooWorkItem
FooWorkManager
FooWorkManagerFactory
OneShotTimer
ResultCollector
ThreadPool
TimerExecutor
TimerManagerService
TimerManagerServiceMBean
WorkManagerService
WorkManagerServiceMBean
|


Unless, scope initialization appears inside any Items control, scope is considered as a global Scope and is referred as $appScope. $appScope is same as $scope for all controls in the same scope. However nested scope controls can access global scope by using $appScope.
As we have come up with a pattern of defining model named global variable for the page, we have included same model in $appScope.
<script type='text/javascript'>
var model = { name: 'Atoms.js' };
</script>
Now, $scope.model in global scope, or $appScope.model refers to exact same variable defined above.
If you have any other global variables that you would like add in the scope, you have to initialize scope as following. Lets say you have a variable named jConfigValues, defined in some javascript.
<script type='text/javascript'>
({ jConfigValues: jConfigValues })
</script>
Note: Please note the difference here, script shown in previous section is a simple javascript defined in page header. Script shown in this example is an Session Initialization Script which only appears within Atoms.js enabled HTML Elements.
In AJAX applications, saving state of page is very important. As in browser, we need back button functionality to function exactly like the way we get with simple web pages. Atoms.js makes it easy to maintain navigation state by serializing global scope (appScope) values into URL separated by # token. Scope initialization is ignored if values exists in URL hash and it works without writing any extra scripts.
Lets review above example.
URL hash changes whenever any scope property is modified by Atoms.js as a result of any User Action or any AJAX result. All the properties of appScope are saved onto URL. Size of URL depends upon browser, Atoms.js has no limit, but we recommend using only smaller values like integer indexes or short keywords to be saved.
You may notice that in above example, we also have a function called 'display' and a list called 'list', but both were not serialized on URL. Here is why, scope values are only saved on URL under following conditions.


JavaScript Review
Cartoon Character List
Character First Name:
Character Last Name:
Cartoon Name:
Additional Information:
Add Cartoon


code:

Ramones: Hey! Ho! Let's Go

This page uses the fallback version of <html5-h>
that extends

. You need only use this version if you are not using HTML5 sectioning elements or if JavaScript may be unavailable.
If you are working entirely in HTML5 and JavaScript will be available, please use the recommended, stand-alone version of <html5-h>
instead. More information can be found in the README.
code:

Heading text level 1

Heading text level 2

Heading text level 3

Heading text level 4

Heading text level 5

Heading text level 6


The purpose of this challenge is to demonstrate the MITRE Top 25 programming flaw: "Cross-Site Request Forgery (CSRF)".
The web application does not, or can not, sufficiently verify whether a well-formed, valid, consistent request was intentionally provided by the user who submitted the request.
CSRF and XSS are related. They are both part of a category of attacks known as Client Side attacks. They are called this way because they occur in the browser and usually have the target the user of the application rather than the server.
If an application only relies on cookies to validate the authentication of a request, it allows CSRF.
Consider the following attack scenario.
Just like in the challenge before you have to exploit the social networking feature where users can see each others profiles.
Use CSRF to cause any user that sees your profile to change their Display Name to Banjo
PS: The developers have introduced a feature to filter out tags and prevent the XSS seen in the previous challenge. You will have to figure out how to send the request to update other users profiles without using HTML markup.


Date of birth:10 may 2006 place of study/work: kariong mountains high school web-site:link to instagram


LAST UPDATED: March 18, 2020
This Privacy Policy explains how information about you is collected, used and disclosed by Kollabo (hereafter “Kollabo”, “our,” “we,” or “us”). This Privacy Policy applies to information we collect when you use our website, mobile applications and other online services (collectively, the “Kollabo Service”) or when you otherwise interact with us.
We may change this Privacy Policy from time to time. If we make changes, we will notify you by revising the date at the top of the policy and, in some cases, we may provide you with more prominent notice (such as adding a statement to our homepage or sending you an email notification). We encourage you to review the Privacy Policy whenever you access the Kollabo Service to stay informed about our information practices and the ways you can help protect your privacy.
If you are a California resident, please see the “Information for California Consumers” section below.
Information You Provide to Us. Kollabo collects information that you provide directly to us. For example, we collect information when you register or update the details of your account, post information in order to sell any good through the Kollabo Service, communicate with other users, provide reviews or other comments, purchase any services from Kollabo, request customer support or otherwise communicate with us. The types of information we may collect include your name, email address, Kollabo account password, postal address including city, state and zip code, photographs, written descriptions of your posted goods, any bids you make on offers through the Kollabo Service, any reviews or comments you make using the Kollabo Service, all correspondence or communications with other users or with us conducted via the Kollabo Service, and any other information you choose to provide.
Information We Collect Automatically When You Use the Kollabo Service. When you access or use the Kollabo Service, we automatically collect information about you, including:
2.1. Log Information: We log information about your use of the Kollabo Service, including the type of browser you use, access times, pages viewed, your IP address and the page you visited before navigating to our Kollabo Service.
2.2. Device Information: We collect information about the computer or mobile device you use to access the Kollabo Service, including the hardware model, operating system and version, unique device identifiers and mobile network information.
2.3. Location Information: If you consent to the collection of location information from your computer or mobile device, we may collect this information when you access or use the Kollabo Service, or when you otherwise consent to the collection of this information. We may also use your IP address to infer an approximate geographic location for your computer or device. For more details, please see “Your Information Choices” below.
5.1. Enable you to access and use the Kollabo Service, to personalize the Kollabo Service, and to optimize the type of offers presented to you when you use the Kollabo Service;
5.2. Create and maintain a trusted and safer environment on the Kollabo Service, including through detection and prevention of fraud, unauthorized access, intrusion, and service interruption;
5.3. Operate, protect, improve and optimize the Kollabo Service, our business, and our users’ experience;
5.4. Perform analytics, conduct research, and monitor and analyze trends, usage and activities in connection with the Kollabo Service;
5.5. Communicate with you about products, services, offers, promotions, rewards, and events offered by Kollabo and others, and provide advertisements that match user profiles or interests, news and information we think will be of interest to you;
5.6. Send you service, support and administrative messages, reminders, technical notices, updates, security alerts, and information requested by you;
5.7. Comply with our legal obligations or requests from law enforcement agencies, resolve any disputes that we may have with any of our users, and enforce our agreements with third parties;
5.8. Carry out any other purpose for which the information was collected.
Kollabo is based in the United States and the information we collect is governed by U.S. law. By accessing or using the Kollabo Service or otherwise providing information to us, you consent to the processing and transfer of information in and to the U.S. and other countries, where you may not have the same rights and protections as you do under local law.
General. We do not share information about you with third-parties such as vendors, consultants and other service providers.
Social Sharing Features. The Kollabo Service may offer social sharing features and other integrated tools (such as the Facebook “Like” button), which let you share actions you take on the Kollabo Service with other media, and vice versa. The use of such features enables the sharing of information with your friends or the public, depending on the settings you establish with the entity that provides the social sharing feature. For more information about the purpose and scope of data collection and processing in connection with social sharing features, please visit the privacy policies of the entities that provide those features.
Account Information. You may update, correct or remove account profile information at any time by logging in to your account. If you would like to cancel your Kollabo account entirely, please contact us, and enter your request for cancellation, but please note that we may retain certain information as required by law or for legitimate business purposes. For example, any reviews, forum postings and similar materials posted by you may continue to be publicly available on the Kollabo Service in association with your account name even after your Kollabo account is cancelled. We may also retain cached or archived copies of information about you for a certain period of time.
Location Information. When you first download or launch any of our mobile applications that collect location information or first use any features that use location information, you will be asked to consent to the application’s collection of this information. If you initially consent to our collection of location information, you can subsequently stop the collection of this information at any time by changing the preferences on your mobile device. You may also stop our collection of location information by following the standard uninstall process to remove all of our mobile applications form your device.
In some features, Kollabo uses mapping services from Google Maps/Google Earth, including Google Maps API(s). Your use of these services is subject to Google’s terms of service.
Promotional Communications. You may opt out of receiving promotional emails, text messages, and other communications from Kollabo by following the instructions in those emails, text messages or other communications or by logging on to your Kollabo account and going to Settings.
Information for California Residents. This section provides additional disclosures required by the California Consumer Privacy Act (“CCPA”), which applies to the personal information we collect about California residents.
In the preceding 12 months, we have collected the following categories of personal information: identifiers (such as name and email address); internet or other electronic network activity information (such as usage data); commercial information (such as products purchased or sold); geolocation data; inferences (such as approximate location); visual or similar information (such as photographs of yourself); and other information that relates to or is reasonably capable of being associated with you and that does not fit neatly into one of the other categories (such as personal information contained in reviews, comments and communications with us). For more details about the personal information we collect, including the categories of sources, please see the [Collection of Information] section above.
We collect personal information for the business and commercial purposes listed in the Use of Information section above. We share the personal information we collect with the categories of third parties listed in the Sharing of Information section above.
Subject to certain limitations, the CCPA provides California residents with the right to:
We do not “sell” the personal information we collect (and will not sell it in the future without providing a right to opt out).
To submit a request to know or a request to delete, please send an email to<EMAIL_ADDRESS>In both cases, we will verify your request by asking you to provide information that matches information we have on file about you. You may designate another person as your authorized agent to act on your behalf with regard to CCPA requests. If you have designated an authorized agent to submit requests on your behalf, we will require proof that the person is authorized to act on your behalf and may also still ask you to verify your identity with us directly.
If you’d like to provide feedback to us about this Privacy Policy, or if you have any questions, please contact us at legal@kollabo.io.


My favorite thing to do in my spare time is play video games. I usually play PC games that I download from Steam, but I also own a Nintendo Switch, which is usually for when I am away from home and wish to play portably. While I prefer a keyboard for first person games, I will use a controller for platformers or simulation games. Sandbox and roguelikes are my personal favorite genres of games.


--- layout: post title: 3 easy improvements in your microservices architecture author: Nicolas Frankel category : talk ---
While microservices offer better scalability, they actually decrease performance and resiliency.
I’ll show 3 areas in which it’s possible to cope with that.
While a microservices architecture is more scalable than a monolith, it has a direct hit on performance.
To cope with that, one performance improvement is to set up a cache.
It can be configured for database access, for REST calls or just to store session state across a cluster of server nodes.
In this demo-based talk, I’ll show how Hazelcast In-Memory Data Grid can help you in each one of those areas and how to configure it. Hint: it’s much easier than one would expect.
Note that this talk will be held in English!
Donnerstag, 06.02.2020, 19:00, Chaos Computer Club Kaiserslautern
Nicolas Frankel is a Developer Advocate with 15+ years experience consulting for many different customers, in a wide range of contexts (such as telecoms, banking, insurances, large retail and public sector). He's usually working on Java/Java EE and Spring technologies, but with focused interests like Rich Internet Applications, Testing, CI/CD and DevOps. Currently, he's working for Hazelcast. He also doubles as a teacher in universities and higher education schools, a trainer and triples as a book author.


ALL PROVINCES
Alberta
British Columbia
Manitoba
New Brunswick
Newfoundland and Labrador
Northwest Territories
Nova Scotia
Nunavut
Ontario
Prince Edward Island
Quebec
Saskatchewan
Yukon
Select a county!