Section 15 Theil-Sen trends
15.1 Trend estimates
Calculating trends for air pollutants is one of the most important and common tasks that can be undertaken. Trends are calculated for all sorts of reasons. Sometimes it is useful to have a general idea about how concentrations might have changed. On other occasions a more definitive analysis is required; for example, to establish statistically whether a trend is significant or not. The whole area of trend calculation is a complex one and frequently trends are calculated with little consideration as to their validity. Perhaps the most common approach is to apply linear regression and not think twice about it. However, there can be many pitfalls when using ordinary linear regression, such as the assumption of normality, autocorrelation etc.
One commonly used approach for trend calculation in studies of air pollution is the non-parametric Mann-Kendall approach (Hirsch, Slack, and Smith 1982). Wilcox (2010) provides an excellent case for using ‘modern methods’ for regression including the benefits of non-parametric approaches and bootstrap simulations. Note also that the all the regression parameters are estimated through bootstrap resampling.
The Theil-Sen method dates back to 1950, but the basic idea pre-dates 1950 (Theil 1950; Sen 1968). It is one of those methods that required the invention of fast computers to be practical. The basic idea is as follows. Given a set of \(n\) \(x\), \(y\) pairs, the slopes between all pairs of points are calculated. Note, the number of slopes can increase by \(\approx\) \(n^2\) so that the number of slopes can increase rapidly as the length of the data set increases. The Theil-Sen estimate of the slope is the median of all these slopes. The advantage of the using the Theil-Sen estimator is that it tends to yield accurate confidence intervals even with non-normal data and heteroscedasticity (non-constant error variance). It is also resistant to outliers — both characteristics can be important in air pollution. As previously mentioned, the estimates of these parameters can be made more robust through bootstrap-resampling, which further adds to the computational burden, but is not an issue for most time series which are expressed either as monthly or annual means. Bootstrap resampling also provides the estimate of \(p\) for the slope.
An issue that can be very important for time series is dependence or autocorrelation in the data. Normal (in the statistical sense) statistics assume that data are independent, but in time series this is rarely the case. The issue is that neighbouring data points are similar to one another (correlated) and therefore not independent. Ignoring this dependence would tend to give an overly optimistic impression of uncertainties. However, taking account of it is far from simple. A discussion of these issues is beyond the aims of this report and readers are referred to standard statistical texts on the issue. In openair we follow the suggestion of Kunsch (1989) of setting the block length to \(n^{1/3}\) where n is the length of the time series.
There is a temptation when considering trends to use all the available data. Why? Often it is useful to consider specific periods. For example, is there any evidence that concentrations of have decreased since 2000? Clearly, the time period used depends on both the data and the questions, but it is good to be aware that considering subsets of data can be very insightful.
Another aspect is that almost all trends are shown as mean concentration versus time; typically by year. Such analyses are very useful for understanding how concentrations have changed through time and for comparison with air quality limits and regulations. However, if one is interested in understanding why trends are as they are, it can be helpful to consider how concentrations vary in other ways. The trend functions in openair do just this. Trends can be plotted by day of the week, month, hour of the day, by wind direction sector and by different wind speed ranges. All these capabilities are easy to use and their effectiveness will depend on the situation in question. One of the reasons that trends are not considered in these many different ways is that there can be a considerable overhead in carrying out the analysis, which is avoided by using these functions. Few, for example, would consider a detailed trend analysis by hour of the day, ensuring that robust statistical methods were used and uncertainties calculated. However, it can be useful to consider how concentrations vary in this way. It may be, for example, that the hours around midday are dominated by heavy vehicle emissions rather than by cars — so is the trend for a pollutant different for those hours compared with say, hours dominated by other vehicle types? Similarly, a much more focussed trend analysis can be done by considering different wind direction, as this can help isolate different source influences.
The TheilSen
function is typically used to determine trends in
pollutant concentrations over several years. However, it can be used
to calculate the trend in any numeric variable. It calculates monthly
mean values from daily, hourly or higher time resolution data, as well
as working directly with monthly means. Whether it is meaningful to
calculate trends over shorter periods of time (e.g. 2 years) depends
very much on the data. It may well be that statistically significant
trends can be detected over relatively short periods but it is another
matter whether it matters. Because seasonal effects can be important
for monthly data, there is the option to deseasonalise the data
first. The timeVariation
function are both useful to determine
whether there is a seasonal cycle that should be removed.
Note also that the symbols shown next to each trend estimate relate to how statistically significant the trend estimate is: \(p\) \(<\) 0.001 = \(\ast\ast\ast\), \(p\) \(<\) 0.01 = \(\ast\ast\), \(p\) \(<\) 0.05 = \(\ast\) and \(p\) \(<\) 0.1 = \(+\).
15.2 Example trend analysis
We first show the use of the TheilSen
function by applying it to
concentrations of O3. The function is called as shown in Figure
15.1.
library(openair)
TheilSen(mydata, pollutant = "o3",
ylab = "ozone (ppb)",
deseason = TRUE,
date.format = "%Y")
## [1] "Taking bootstrap samples. Please wait."
Because the function runs simulations to estimate the uncertainty in the slope, it can take a little time for all the calculations to finish. These printed results show that in this case the trend in O3 was +0.38 units (i.e. ppb) per year as an average over the entire period. It also shows the 95% confidence intervals in the trend ranged between 0.21 to 0.51 ppb/year. Finally, the significance level in this case is very high; providing very strong evidence that concentrations of O3 increased over the period. The plot together with the summary results is shown in Figure 15.1. Note that if one wanted to display the confidence intervals in the slope at the 99% confidence intervals, the code would be Figure 15.2.
TheilSen(mydata, pollutant = "o3", ylab = "ozone (ppb)", alpha = 0.01)
Sometimes it is useful to consider a subset of data, perhaps by
excluding some years. This is easy with the filter
function. The
following code calculates trends for years greater than 1999 i.e. from
2000 onward.
TheilSen(filter(mydata, format(date, "%Y") > 1999),
pollutant = "o3",
ylab = "ozone (ppb)")
It is also possible to calculate trends in many other ways e.g. by
wind direction. Considering how trends vary by wind direction can be
extremely useful because the influence of different sources invariably
depends on the direction of the wind. The TheilSen
function
splits the wind direction into 8 sectors i.e. N, NE, E etc. The
Theil-Sen slopes are then calculated for each direction in turn. This
function takes rather longer to run because the simulations need to be
run eight times in total. Considering concentrations of O3 again,
the output is shown in Figure 15.2. Note that this plot is
specifically laid out to assist interpretation, with each panel
located at the correct point on the compass. This makes it easy to see
immediately that there is essentially no trend in O3 for southerly
winds i.e. where the road itself has the strongest influence. On the
other hand the strongest evidence of increasing O3 are for
northerly winds, where the influence of the road is much less. The
reason that there is no trend in O3 for southerly winds is that
there is always a great excess of NO, which reacts with O3 to form
NO2. At this particular location it will probably take many more
years before O3 concentrations start to increase when the wind
direction is southerly. Nevertheless, there will always be some
hours that do not have such high concentrations of NO.
TheilSen(mydata, pollutant = "o3", type = "wd",
deseason = TRUE,
date.format = "%Y",
ylab = "ozone (ppb)")
## [1] "Taking bootstrap samples. Please wait."
## [1] "Taking bootstrap samples. Please wait."
## [1] "Taking bootstrap samples. Please wait."
## [1] "Taking bootstrap samples. Please wait."
## [1] "Taking bootstrap samples. Please wait."
## [1] "Taking bootstrap samples. Please wait."
## [1] "Taking bootstrap samples. Please wait."
## [1] "Taking bootstrap samples. Please wait."
The option slope.percent
can be set to express slope
estimates as a percentage change per year. This is useful for
comparing slopes for sites with very different concentration levels
and for comparison with emission inventories. The percentage change
uses the concentration at the beginning and end months to express the
mean slope.
The trend, \(T\) is defined as:
\[ T [\%.yr^{-1}] = 100.\left(\frac{C_{End}}{C_{Start}} - 1\right)\Bigg /N_{years} \tag{15.1} \]
where \(C_{End}\) and \(C_{Start}\) are the mean concentrations for the end and start date, respectfully. \(N_{years}\) is the number of years (or fractions of) the time series spans.
TheilSen(mydata, pollutant = "o3", deseason = TRUE,
slope.percent = TRUE)
The TheilSen
function was written to work with hourly data,
which is then averaged into monthly or annual data. However, it is
realised that users may already have data that is monthly or
annual. The function can therefore accept as input monthly or annual
data directly. However, it is necessary to ensure the date field
is in the correct format. Assuming data in an Excel file in the
format dd/mm/YYYY (e.g. 23/11/2008), it is necessary to convert this
to a date format understood by R, as shown below. Similarly, if annual
data were available, get the dates in formats like ‘2005-01-01,’
‘2006-01-01’ … and make sure the date is again formatted using
as.Date
. Note that if dates are pre-formatted as YYYY-mm-dd,
then it is sufficient to use as.Date
without providing any
format information because it is already in the correct format.
$date = as.Date(mydata$date, format = "%d/%m/%Y") mydata
Finally, the TheilSen
function can consider trends at
different sites, provided the input data are correctly formatted. For
input, a data frame with three columns is required: date, pollutant
and site. The call would then be, for example:
TheilSen(mydata, pollutant = "no2", type = "site")
15.2.1 Output
The TheilSen
function provides lots of output data for further
analysis or adding to a report. To obtain it, it is necessary to read
it into a variable:
<- TheilSen(mydata, pollutant = "o3", deseason = TRUE, type = "wd") MKresults
## [1] "Taking bootstrap samples. Please wait."
## [1] "Taking bootstrap samples. Please wait."
## [1] "Taking bootstrap samples. Please wait."
## [1] "Taking bootstrap samples. Please wait."
## [1] "Taking bootstrap samples. Please wait."
## [1] "Taking bootstrap samples. Please wait."
## [1] "Taking bootstrap samples. Please wait."
## [1] "Taking bootstrap samples. Please wait."
This returns a list of two data frames containing all the monthly mean values and trend statistics and an aggregated summary. The first 6 lines are shown next:
head(MKresults$data[[1]])
## wd date conc a b upper.a upper.b lower.a
## 1 E 1998-01-01 5.552253 -2.825001 0.0007185124 -8.826666 0.001230109 3.848278
## 2 E 1998-02-01 2.919639 -2.825001 0.0007185124 -8.826666 0.001230109 3.848278
## 3 E 1998-03-01 3.849363 -2.825001 0.0007185124 -8.826666 0.001230109 3.848278
## 4 E 1998-04-01 4.051668 -2.825001 0.0007185124 -8.826666 0.001230109 3.848278
## 5 E 1998-05-01 2.304686 -2.825001 0.0007185124 -8.826666 0.001230109 3.848278
## 6 E 1998-06-01 -1.560438 -2.825001 0.0007185124 -8.826666 0.001230109 3.848278
## lower.b p p.stars slope intercept intercept.lower
## 1 0.0001449688 0.01669449 * 0.262257 -2.825001 3.848278
## 2 0.0001449688 0.01669449 * 0.262257 -2.825001 3.848278
## 3 0.0001449688 0.01669449 * 0.262257 -2.825001 3.848278
## 4 0.0001449688 0.01669449 * 0.262257 -2.825001 3.848278
## 5 0.0001449688 0.01669449 * 0.262257 -2.825001 3.848278
## 6 0.0001449688 0.01669449 * 0.262257 -2.825001 3.848278
## intercept.upper lower upper slope.percent lower.percent
## 1 -8.826666 0.05291362 0.4489896 5.79801 0.9925881
## 2 -8.826666 0.05291362 0.4489896 5.79801 0.9925881
## 3 -8.826666 0.05291362 0.4489896 5.79801 0.9925881
## 4 -8.826666 0.05291362 0.4489896 5.79801 0.9925881
## 5 -8.826666 0.05291362 0.4489896 5.79801 0.9925881
## 6 -8.826666 0.05291362 0.4489896 5.79801 0.9925881
## upper.percent
## 1 11.9614
## 2 11.9614
## 3 11.9614
## 4 11.9614
## 5 11.9614
## 6 11.9614
Often only the trend statistics are required and not all the monthly values. These can be obtained by:
$data[[2]] MKresults
## wd p.stars date conc a b upper.a
## 1 E * 2001-09-15 5.989974 -2.825001 7.185124e-04 -8.8266663
## 2 N *** 2001-09-15 9.786267 -19.142102 2.485907e-03 -28.7157702
## 3 NE *** 2001-09-15 9.728994 -9.728741 1.624970e-03 -24.8702270
## 4 NW *** 2001-09-15 9.786755 -12.940875 1.937119e-03 -22.4192235
## 5 S 2001-09-15 5.052728 4.472516 2.509968e-05 1.0744367
## 6 SE 2001-09-15 5.780645 4.822713 7.357549e-05 0.8230425
## 7 SW 2001-09-15 4.761100 1.818359 2.444766e-04 -1.9192744
## 8 W ** 2001-09-15 5.618727 -1.178793 5.736058e-04 -5.6090971
## upper.b lower.a lower.b p slope intercept
## 1 0.0012301086 3.848278 1.449688e-04 0.016694491 0.262257022 -2.825001
## 2 0.0032893251 -9.725509 1.649012e-03 0.000000000 0.907355964 -19.142102
## 3 0.0029640572 3.185458 5.003924e-04 0.000000000 0.593114074 -9.728741
## 4 0.0027690442 -3.396506 1.122414e-03 0.000000000 0.707048447 -12.940875
## 5 0.0003281521 7.156572 -2.081018e-04 0.868113523 0.009161382 4.472516
## 6 0.0004318379 8.727162 -2.572925e-04 0.684474124 0.026855053 4.822713
## 7 0.0005716588 5.538235 -8.999974e-05 0.110183639 0.089233973 1.818359
## 8 0.0009552189 3.041723 2.044787e-04 0.003338898 0.209366110 -1.178793
## intercept.lower intercept.upper lower upper slope.percent
## 1 3.848278 -8.8266663 0.05291362 0.4489896 5.7980099
## 2 -9.725509 -28.7157702 0.60188926 1.2006037 14.4454302
## 3 3.185458 -24.8702270 0.18264323 1.0818809 8.6085473
## 4 -3.396506 -22.4192235 0.40968128 1.0107011 10.2917646
## 5 7.156572 1.0744367 -0.07595715 0.1197755 0.1937191
## 6 8.727162 0.8230425 -0.09391177 0.1576208 0.4816904
## 7 5.538235 -1.9192744 -0.03284991 0.2086555 2.0662607
## 8 3.041723 -5.6090971 0.07463472 0.3486549 4.4665024
## lower.percent upper.percent
## 1 0.9925881 11.961401
## 2 8.4310805 24.381910
## 3 2.1997335 19.875875
## 4 5.0687904 17.131133
## 5 -1.5105886 2.703463
## 6 -1.5405900 3.008348
## 7 -0.7113745 5.313247
## 8 1.4540385 8.381275
In the results above the lower
and upper
fields provide the
95% (or chosen confidence interval using the alpha
option) of
the trend and slope
is the trend estimate expressed in units/year.


fitzdarcy@home:~$ darcy --home
Hello, it's Fitzwilliam Darcy. I'm the famous Jane Austen character from
the book Pride & Prejudice. Please take a look at the manual to see how to
use this site and learn more about me.
In order to check out any one of these options below, please type the corresponding command to
the command line and hit enter. DO NOT include braces < >. Hit tab to focus on the command line.
You can switch to Graphical User Interface (with buttons, clickable links and stuff)
by clicking the little button on the top.(Coming soon)
Example:
fitzdarcy@home:~$ darcy --quotes
--------------------------------------------------


Name  :  Abhinav Harsh
Roll no  :   1906003
What are HTML tags
HTML tags are like keywords which defines that how web browser will format and display the content. With the help of tags, a web browser can distinguish between an HTML content and a simple content. HTML tags contain three main parts: opening
tag, content and closing tag. But some HTML tags are unclosed tags.
When a web browser reads an HTML document, browser reads it from top to bottom and left to right. HTML tags are used to create HTML documents and render their
properties. Each HTML tags have different properties. An HTML file must have some essential tags so that web browser can differentiate between a simple text and HTML text. You can use as many tags you want as per your code requirement.
Unclosed HTML Tags
Some HTML tags are not closed, for example br and hr.
< br>
Tag: br stands for break line, it breaks the line of the code.
< hr>
Tag: hr stands for Horizontal Rule. This tag is used to put a line across the webpage.
HTML Image and Object Tags
< img> tag: used to embed an image in a web page. Images are not technically inserted into a web page; images are linked to web pages. The < img> tag creates a holding space for the referenced image. The < img> tag is empty, it contains attributes only, and does not have a closing tag.
< area> tag: The < area> tag defines an area inside an image map (an image map is an image with clickable areas). < area> elements are always nested inside a < map> tag.
< map> tag: used to define an image map. An image map is an image with clickable areas. The required name attribute of the < map> element is associated with the < img>'s usemap attribute and creates a relationship between the image and the map.
< param> tag: The < param> tag is used to define parameters for an < object> element.
< object> tag: defines a container for an external resource. The external resource can be a web page, a picture, a media player, or a plug-in application. To embed a picture, it is better to use the < img> tag. To embed HTML, it is better to use the < iframe> tag.
HTML Form Tags
< form> tag: used for creating a form for user input. A form can contain textfields, checkboxes, radio-buttons and more. Forms are used to pass user-data to a specified URL.
< input> tag: used to create interactive controls for web-based forms in order to accept data from the user; a wide variety of types of input data and control widgets are available, depending on the device and user agent
< textarea> tag: defines a multi-line text input control. The < textarea> element is often used in a form, to collect user inputs like comments or reviews. A text area can hold an unlimited number of characters, and the text renders in a fixed-width font (usually Courier).
< select> tag: used to create a drop-down list. The < select> element is most often used in a form, to collect user input. The name attribute is needed to reference the form data after the form is submitted
< option> tag: used to define an item contained in a < select> , an < optgroup> , or a < datalist> element. As such, < option> can represent menu items in popups and other lists of items in an HTML document.
< optgroup> tag: The < optgroup> tag is used to group related options in a < select> element (drop-down list).
< button> tag: The < button> tag defines a clickable button. Inside a < button> element you can put text (and tags like < i>, < b>, < strong>, < br>, < img>, etc.). That is not possible with a button created with the < input> element!
< label> tag: It defines a text label for the input field of form. Users who have difficulty clicking on very small regions (such as checkboxes) - because when a user clicks the text within the < label> element, it toggles the input (this increases the hit area).
< fieldset> tag: used to group several controls as well as labels ( < label> ) within a web form.
< legend> tag: he < legend> tag defines a caption for the < fieldset> element.


Hello my Steemit friends,
The Partiko Mascot Design Contest is over and I am happy to see many Steemians from different counties joined the contest. Let me announce the final winners selected by @partiko, we are so blown away by the the quality and talent of all the submissions! Thanks for joining and your support towards @partiko. Hope that you have enjoyed the contest and had fun! Every participant is also eligible to receive 1000 Partiko Points, and you may read more about the details here. Let’s enjoy the masterpieces created by the winners:
1st Prize:
https://steemit.com/partiko-contest/@isakost/my-little-dragon--for-partiko-mascot-design-contest-443aucbp by @isakost
2nd Prize:
https://steemit.com/partiko-contest/@yoshiko/entry-for-partiko-mascot-design-contest-partiko by @yoshiko
3rd Prize:
https://steemit.com/partiko-contest/@angelina6688/design-a-mascot-for-partikolovely-crab1partiko-gotl7rl7 by @angelina6688
Merit Prize:
https://steemit.com/partiko-contest/@djynn/entry-for-partiko-mascot-design-contest--partiko-3g8wt7rg by @djynn
https://steemit.com/partiko-contest/@ishanvirtue/partiko-mascot-design-contest--teeko-rems9etb by @ishanvirtue
https://steemit.com/partiko-contest/@remyrequenart/partiko-mascot--design-contest-7nro5jo6 by @remyrequenart
https://steemit.com/art/@laykenia/partiko-mascot-desing-contest-x3xzutmi by @laykenia
https://steemit.com/spanish/@marylucy/mi-propuesta-para-el-concurso-de-la-mascota-de-partiko-orqmqul3 by @marylucy
Congratulations to all the winners. All the SBD rewards have been sent directly to your wallets, cheers and enjoy your day.
At last, if you like their drawings, don’t forget to support and follow them 🙂
Posted using Partiko iOS
This page is synchronized from the post: ‘Partiko Mascot Design Contest | Announcing the winners’


Class Base
Declares the base for a parse action that extract and generates definition files.
- Peg\Custom\CommandLine\Action
- Peg\Custom\Command\Action\Parse\Base
Direct known subclasses
Peg\Custom\Command\Action\Parse\Doxygen
public
|
#
__construct( string $input_format )
You derived class should override this and set the input_format name of your parser/lexer so it is called apropiately when this variable is set. |
public
|
#
OnCall(
You shouldn't override this method, instead write a Start() implementation. |
abstract public
|
protected
|
$command |
|
#
Reference to command that called this action. |
protected
string
|
$input_format |
|
#
Format of the files to parse/lex. |
protected
string
|
$output_format |
|
#
Format used to store the parsed definition files. |
protected
string
|
$headers_path |
|
#
This is used optionally by a children implementing this class in order for them to correctly resolve a header file. For example: doxygen xml files store the full path to the header file where a symbol was found, eg: /home/user/libs/wx/frame.h, if this variable is set to /home/user/libs/ the final header file will be stored on the symbols object as wx/frame.h |
protected
boolean
|
$verbose |
|
#
Flag that indicates if the lexer/parser should output messages of its current status. |