--- layout: post title: Chilled tags: [] status: publish type: post published: true meta: {} ---
Sorry about the lack of posts over the last few days, but I took off for the weekend and hit the Big Chill festival. which was absolutely awesome. If you decide to go to a only one festival, this is the one!! Three days of sun, music, food, beer, croquet, relaxation and camping in the English country side!!!


National Cyber Security Awareness Month was established to strengthen the weakest point of any security solution: humans. No matter how effective—or expensive—the security tools protecting your network are, there’s no way to predict the damage caused by a single careless user. The war against cyber criminals is fought each time a user decides to click an unfamiliar link or open an attachment—and just a single mistake could be the reason for massive data loss.
To help out IT security managers during this Cyber Security Awareness Month, we’ve compiled a list of things your users should be thinking about whenever they’re using the Internet. They’ve probably heard many or all of these tips before, but repetition doesn’t spoil the prayer. If you take security seriously, you’re already using Umbrella by OpenDNS to protect your network from malware, botnets, and other advanced threats. And if you’re smart, you know that there’s no substitute for educating your users. Share this list with your users, this month—and every month.  Don’t ever say “It won’t happen to me.”
Practice good password management. Use a strong mix of characters, and don’t use the same password for multiple sites. Don’t share your password with others, don’t write it down, and definitely don’t write it on a post-it note attached to your monitor.
Never leave your devices unattended. If you need to leave your computer, phone, or tablet for any length of time—no matter how short—lock it up so no one can use it while you’re gone. If you keep sensitive information on a flash drive or external hard drive, make sure to lock it up as well.
Always be careful when clicking on attachments or links in email. If it’s unexpected or suspicious for any reason, don’t click on it. Double check the URL of the website the link takes you to: bad actors will often take advantage of spelling mistakes to direct you to a harmful domain. Think you can spot a phony website? Try our Phishing Quiz.
Sensitive browsing, such as banking or shopping, should only be done on a device that belongs to you, on a network that you trust. Whether it’s a friend’s phone, a public computer, or a cafe’s free WiFi—your data could be copied or stolen.
Back up your data regularly, and make sure your anti-virus software is always up to date.
Be conscientious of what you plug in to your computer. Malware can be spread through infected flash drives, external hard drives, and even smartphones.
Watch what you’re sharing on social networks. Criminals can befriend you and easily gain access to a shocking amount of information—where you go to school, where you work, when you’re on vacation—that could help them gain access to more valuable data.
Offline, be wary of social engineering, where someone attempts to gain information from you through manipulation. If someone calls or emails you asking for sensitive information, it’s okay to say no. You can always call the company directly to verify credentials before giving out any information.
Be sure to monitor your accounts for any suspicious activity. If you see something unfamiliar, it could be a sign that you’ve been compromised.


2 Samuel 2:7
Therefore now let your hands be strengthened, and be ye valiant: for your master Saul is dead, and also the house of Judah have anointed me king over them.


import logging
import telebot
import time
import helper
import edit
import history
import display
import estimate
import delete
import add
import budget
from datetime import datetime
from jproperties import Properties
configs = Properties()
with open('user.properties', 'rb') as read_prop:
configs.load(read_prop)
api_token = str(configs.get('api_token').data)
bot = telebot.TeleBot(api_token)
telebot.logger.setLevel(logging.INFO)
option = {}
Define listener for requests by user
def listener(user_requests):
listener(user_requests): Takes 1 argument user_requests and logs all user interaction with the bot including all bot commands run and any other issue logs.
for req in user_requests:
if(req.content_type == 'text'):
print("{} name:{} chat_id:{} \nmessage: {}\n".format(
str(datetime.now()), str(req.chat.first_name), str(req.chat.id), str(req.text)))
message = "Sorry, I can't understand messages yet :/\n" + \
"I can only understand commands that start with /. \n\n" + \
"Type /faq or /help if you are stuck."
try:
helper.read_json()
global user_list
chat_id = user_requests[0].chat.id
if user_requests[0].text[0] != '/':
bot.send_message(chat_id, message)
except:
pass
bot.set_update_listener(listener)
@bot.message_handler(commands=['help'])
def help(m):
helper.read_json()
global user_list
chat_id = m.chat.id
message = 'Here are the commands you can use: \n'
commands = helper.getCommands()
for c in commands:
message += "/" + c + ", "
message += commands[c] + “\n\n”
message += '\nUse /menu for detailed instructions about these commands.'
bot.send_message(chat_id, message)
@bot.message_handler(commands=['faq'])
def faq(m):
helper.read_json()
global user_list
chat_id = m.chat.id
faq_message = '"What does this bot do?"\n' + \
'>> DollarBot lets you manage your expenses so you can always stay on top of them! \n\n' + \
'"How can I add an epxense?" \n' + \
'>> Type /add, then select a category to type the expense. \n\n' + \
'"Can I see history of my expenses?" \n' + \
'>> Yes! Use /display to get a graphical display, or /history to view detailed summary.\n\n' + \
'"I added an incorrect expense. How can I edit it?"\n' + \
'>> Use /edit command. \n\n' + \
'"Can I check if my expenses have exceeded budget?"\n' + \
'>> Yes! Use /budget and then select the view category. \n\n'
bot.send_message(chat_id, faq_message)
defines how the /start and /help commands have to be handled/processed
@bot.message_handler(commands=['start', 'menu'])
def start_and_menu_command(m):
start_and_menu_command(m): Prints out the the main menu displaying the features that the bot offers and the corresponding commands to be run from the Telegram UI to use these features. Commands used to run this: commands=[‘start’, ‘menu’]
helper.read_json()
global user_list
chat_id = m.chat.id
print(‘receieved start or menu command.’) text_into = “Welcome to the Dollar Bot!”
text_intro = "Welcome to the Dollar Bot! \n" + \
"DollarBot can track all your expenses with simple and easy to use commands :) \n" + \
"Here is the complete menu. \n\n"
“Type /faq or /help to get stated.”
commands = helper.getCommands()
for c in commands: # generate help text out of the commands dictionary defined at the top
text_intro += "/" + c + ": "
text_intro += commands[c] + "\n\n"
bot.send_message(chat_id, text_intro)
return True
defines how the /new command has to be handled/processed
@bot.message_handler(commands=['add'])
def command_add(message):
command_add(message) Takes 1 argument message which contains the message from the user along with the chat ID of the user chat. It then calls add.py to run to execute the add functionality. Commands used to run this: commands=[‘add’]
add.run(message, bot)
function to fetch expenditure history of the user
@bot.message_handler(commands=['history'])
def command_history(message):
command_history(message): Takes 1 argument message which contains the message from the user along with the chat ID of the user chat. It then calls history.py to run to execute the add functionality. Commands used to run this: commands=[‘history’]
history.run(message, bot)
function to edit date, category or cost of a transaction
@bot.message_handler(commands=['edit'])
def command_edit(message):
command_edit(message): Takes 1 argument message which contains the message from the user along with the chat ID of the user chat. It then calls edit.py to run to execute the add functionality. Commands used to run this: commands=[‘edit’]
edit.run(message, bot)
function to display total expenditure
@bot.message_handler(commands=['display'])
def command_display(message):
command_display(message): Takes 1 argument message which contains the message from the user along with the chat ID of the user chat. It then calls display.py to run to execute the add functionality. Commands used to run this: commands=[‘display’]
display.run(message, bot)
function to estimate future expenditure
@bot.message_handler(commands=['estimate'])
def command_estimate(message):
estimate.run(message, bot)
handles “/delete” command
@bot.message_handler(commands=['delete'])
def command_delete(message):
command_delete(message): Takes 1 argument message which contains the message from the user along with the chat ID of the user chat. It then calls delete.py to run to execute the add functionality. Commands used to run this: commands=[‘display’]
delete.run(message, bot)
@bot.message_handler(commands=['budget'])
def command_budget(message):
budget.run(message, bot)
not used
def addUserHistory(chat_id, user_record):
global user_list
if(not(str(chat_id) in user_list)):
user_list[str(chat_id)] = []
user_list[str(chat_id)].append(user_record)
return user_list
main() The entire bot’s execution begins here. It ensure the bot variable begins polling and actively listening for requests from telegram.
def main():
try:
bot.polling(none_stop=True)
except Exception as e:
logging.exception(str(e))
time.sleep(3)
print("Connection Timeout")
if __name__ == '__main__':
main()


@(ticket: Ticket)
@main("View Ticket"){
ID: @ticket.ticket_id
Name: @ticket.name
Description: @ticket.description
Date Created: @ticket.date_created
Last Updated: @ticket.last_updated
Category: @ticket.category
Priority: @ticket.priority
State: @ticket.state
},


Please click below to see the HTML version of this email.
http://shop.cvsphoto.com/r/D435PEA/DBYNE/KMDECT/5CX49P4/E6OH8/20/t?a=D435PEA&b=P4MG2&c=YCYVWLT&d=YR1HIM&e=1&f=61ba4782e8ca0dafaf18
****************************************************
CVS/pharmacy® Photo
****************************************************
BOGO Books, 20% Off Calendars & 25% Off Cases >
http://shop.cvsphoto.com/r/D435PEA/DBYNE/KMDECT/5CX49P4/9THPT/20/t?a=D435PEA&b=P4MG2&c=YCYVWLT&d=YR1HIM&e=1&f=61ba4782e8ca0dafaf18
****************************************************
Add a minimum of two (2) Photo Books to your cart, enter promo code BOGOBOOK and get your second Photo Book of equal or lesser value for FREE. Offer valid online only. Offer ends 1/19/13.
Add any Calendar(s) to your shopping cart, enter promo code CALENDAR20 and get 20% OFF your Calendar(s) order. Offer valid online only. Promotion ends 1/19/13.
Add any Canvas(es) to your shopping cart, enter promo code CANVAS25 and get 25% OFF your Canvas(es) order. Offer valid online only. Promotion ends 1/19/13.
You are receiving this email because you indicated you wanted to be informed of the latest news and specials from CVSphoto.com. If you prefer not to receive future emails from
CVSphoto.com, please unsubscribe below. Please note, if you have subscribed to receive other CVS email communications you will continue to receive them unless you unsubscribe specifically for those.
http://shop.cvsphoto.com/r/D435PEA/DBYNE/KMDECT/5CX49P4/Y9SYF/20/t?a=http://shop.cvsphoto.com/u/D435PEA/DBYNE/KMDECT/5CX49P4/IE/aHVibGVlcGg2MDZAdHVtYmxyLmNvbQ==/
© Copyright 2013 CVSphoto.com
http://shop.cvsphoto.com/r/D435PEA/DBYNE/KMDECT/5CX49P4/NM52K/20/t?a=PhotoEM_11612
One CVS Drive, Woonsocket, RI 02895
[[D435PEA-P4MG2-DBYNE-KMDECT-5CX49P4-M-M2-20130116-555e6be719aba74]]


Askin Techlabs is solving major pain staking problems in different industries by developing Android and iOS applications for our clients. We do this by optimising the effort, accuracy and time function to the greatest extent while providing unique insights into the data that is generated by using our apps. All of this at extremely competitive costs.
By ascertaining whether products manufactured in your facility meet predefined thresholds and are within accepted manufacturing tolerances, to digitally logging all steps of the quality control process, Qoti presents an astute tool for real time quality unit assessment and tracking.
Adit Sharda
The affinity to code never really left Adit’s realm. Post a decade of working in the finance industry and practising as a Chartered Accountant, Adit finds solace in software development and gets his strong code and troubleshooting prowess to the team.
Shantanu Bamnavat
Shantanu brings to Askin a mindset stemming from the reliable and the dependable. He guides the company identifying new responsibilities within a wide range of areas. He has active and dynamic approach to work and getting things done.
Kunal Khatri
A serial entrepreneur, Kunal founded WEC two years ago, a leading automotive parts manufacturer.
He has a strong knack for identifying untapped opportunities and regularly invests in new business ventures. In his free time, he loves travelling the world and trying his hand on new cuisines.


The first time i experieced the 'hug drug' i was a little skeptical about what effect would be on me. I was 19 years old, 105lbs and freshman in college. My boyfiend and his friends had tried it many times before so i was the only one that was going to be a first timer.
An hour before we dropped the pills we made a trip to walmart for some 'goodies': Glowsticks, vicks inhalers, dusters, fruit drinks, and pacifiers. We were at first going to be heading off to a rave but, with it being my first time, i wanted to stay indoors at a friends house to be more comfortable. We had couches set up all around and pillows to sit on. Along with blacklights, strobelights and glowsticks, the atmosphere was very pleasent. The guys kindof brought the rave to me.
After 20 min of taking the pill i still wasnt feeling anything significant, yet everyone else there seemed to be already blowing up. Everyone there had told me that if i stayed happy it would help it kick in. Finally about 45 minutes into it, something came over me. I felt a tingly feeling in my legs, then my chest, and all over arms and hands. I called out for one of the guys there to come over and touch me. I dont know why i wanted to be touched or held..i just did.
 The feeling i experienced at that moment was incredible. I couldn't stop taking immense breaths of air. I layed there for about, what seemed to me, about 20 minutes until i suddenly felt the urge to get up and walk around. Everything i felt for the next hour was amazing.. big bear hugs, ice on my skin, the carpet, and my personal favorite.. my white fuzzy sweater i brought along to keep me warm.
About 3 hours after we dropped I had the best experience of the whole night.  The feelings of warmth from the heater in a car..and then letting the windows down to feel the wind against my skin brought me the greatest feeling of freedom and euphoria. My mind went very clear as i felt the wind brush against my face, I felt as if the earth was showing me its true meaning. Even now, i seem to ride around with the windows down because i still get the feeling of inner warmth.
Along with all the physical feelings that night, I had an emotional experience too. That night I loved everyone. I got closer to one of my boyfriends friends that before that night i had hated. I wanted everyone to be feeling the same amazing feelings i was. I couldnt stop myself from giving massages and kissing people on the cheek. Like many people had told me, my feelings werent sexual at all. I wanted to be loved and give love in return... true love.  Which at the time I wasnt ready for.
There are alot of feelings that came out when I was on E that I didn't even know were inside me. I told people I didnt even know that I would love them forever and keep them in my heart always, which now when i think about it isnt true. I dont even remember who half of them were. But I can say this: The people that were there that night that i still see now and then are closer to me than ever. I am still with my boyfriend, deeply in love, and am best friends with his friend that I used to hate.
 But i can say that I had one of the best night's sleep i think i have ever had.
Ecstasy is something that can change your life. I have only taken it twice but both times have been very rewarding. I advise that if you are going to be a first timer, do it with someone that knows what to expect so that they can help you have a better roll. Also make sure you buy gum or pacifiers! If not your teeth and jaws with be hurting for a couple days after your roll.
I can say that because of ecstasy, I am more open to people, more trusting and i have a new way of looking at life. The world has become a beautiful, interesting, peaceful place to me. I also have a new appreciation for techno music. I never know when my next rolling experience will be.. but I am hoping that it will bring me as much 'ecstasy' as the rest.
Experience Reports are the writings and opinions of the individual authors who submit them.
Some of the activities described are dangerous and/or illegal and none are recommended by Erowid.


--- date: "2016-04-09T16:50:16+02:00" title: Multiple Regression output: learnr::tutorial weight: 3 ---
In the previous section we learnt about simple linear regression as a very simple tool for predicting quantitative measured response variables. Its simple application, easy interpretation and predictive capabilities make it one of the most popular approaches for supervised learning. Developing good understanding of linear regression serves as a solid base for learning how to use and adopt other more sophisticated modelling procedures in the context of machine learning.
Additive Model
The basic idea in regression modelling is that there is a phenomenon of interest whose behaviour we seek to account for or explain. For example a company may be interested in why its sales figures have followed a particular pattern; Government officials may be interested in explaining the behaviour of unemployment statistics.
Let the phenomenon of interest be denoted by \(Y\); as discussed in the previous sections, \(Y\) is also known as the response variable, or the dependent variable. The fundamental nature of \(Y\) is that it displays variability and it is this inherent variability that the data analyst seeks to explain by using regression modelling. Intuitively, if we can account for a large amount of the movements in \(Y\) then in some sense this is good. Conversely, if we put forward an argument which explains only a very small amount of the behaviour of the phenomenon of interest then in some sense this is bad.
In trying to explain, or account for the behaviour of \(Y\) we often build a regression model. As a first step in building such a model we specify a set of variables, called the explanatory or predictor variables, that we believe to be important in explaining the behaviour of, or the variability in \(Y\). Specifying this set of variables is, in effect, the first step in developing as the data analysts our viewpoint, our model, our theory. As such we are entitled to ask where such a viewpoint comes from - for example, it may be the logical outcome of some theoretical argument or it may be a replication of a previous study.
{{% notice warning %}} 💡 We do well to remember that this statement of a model is a viewpoint, a belief, a theory and need not be correct, i.e. true. {{% /notice %}}
Indeed, it is the very essence of the regression model validation to judge if a viewpoint, a belief, a theory is in any sense acceptable. So, when developing a regression model all we argue is: \(Y = f(X_1, X_2, X_3, … , X_k)\), which represents a belief that the response variable \(Y\) depends upon a set of \(k\) explanatory variables \((X_1, X_2, X_3,… , X_k)\).
Specifying a set of variables is not in itself a complete model. We have to be prepared to indicate and argue more precisely how the explanatory variables are supposed to relate to \(Y\): that is, we must specify the functional form that links the set of variables to \(Y\). The simplest relationship is a linear relationship or a straight line. As we have already learnt for a single explanatory variable, \(X_1\), this is given by:
and for a set of \(k\)-variables this is given by:
The linear models specified so far are exact or deterministic. Such structures do not represent statistical problems. The very nature of statistical data is that there is inherent variability in that there is some part of the responses variable that we cannot explain. Thus there is a random element whose behaviour is by nature and definition unpredictable. We know that a random component is going to impact on \(Y\), but we do not know the size or sign of such random happenings.
This random/stochastic part of the model is captured as follows:
\(Y = b_0 + b_1X_1 + e\), for a single explanatory variable model, and
\(Y = b_0 + b_1X_1 + b_2X_2 + ... + b_kX_k + e\), for the general \(k\) explanatory variable model
where \(e\) is also known as the error term.
{{% notice warning %}} 💡 These expressions need to be fully understood and the role and significance of the random/stochastic element has to be fully recognised. {{% /notice %}}
In order to complete the specification of the statistical regression model some assumption has to be made about the underlying process that produces \(e\). The standard approach is to adopt the following distribution structure:
\(e \sim N(0,\sigma^2)\), with the error term from a normal distribution with a mean of \(0\), and a variance of \(\sigma^2\).
Knowing this, we should now be in a position to understand and identify the structure of the standard multiple linear regression model.
Prior Knowledge
As a statement of belief, the specified model just suggests a list of important explanatory variables within a linear structure. We need to be willing to take an extra conceptual step by specifying our expectation on how the individual explanatory variables are connected to the response variable. We should make assertions about:
Again knowledge of these two potential tricky areas are connected to theoretical arguments and are often the results of previous similar studies. Such knowledge is sometimes called prior knowledge, or prior views. In order to give a viewpoint some evidential or empirical support is required:
That is, the proposed theory has to be tested against the available evidence.
{{% notice note %}} It needs to be fully understood that the model developed so far is no more than a belief. {{% /notice %}}
We may believe passionately that \(Y\) is caused by the structure put forward but we do not know the structure in that we do not know the values of the \(b\) parameters. These \(b\) values are held to exist and have particular true numerical values which we need to find out. In an attempt to give our theory/model/viewpoint empirical content we have to collect a sample of pertinent data. On the basis of the information contained in this particular data, we have to estimate the \(b\) values and draw valid conclusions about the nature of the true but unknown underlying structure which is claimed to be responsible for the behaviour of \(Y\).
Building a Model
The sample, of size \(n\), evidence is seen as coming from the following structure:
\[Y_i = b_0 + b_1X_{i1} + b_2X_{i2} + b_3X_{i3} + … + b_kX_{ik} + e_i, \;\;\;\;\; where \;\; i=1, 2, ...n\] Selecting a sample by definition means that we have imperfect information, that is we do not have information on the whole population or we do not see the whole picture. We only see a possibly disjointed part of the whole picture and thus taking a sample creates obvious difficulties in making sense of what we can see. Taking a sample introduces an interpretation problem of trying to say something sensible about the whole picture, the true model, from a part of the picture.
The data handling process involves two major steps, namely:
Model Estimation
As we have discussed, the model developed so far is a viewpoint. It represents our view as to the true underlying structure of the world. We believe that this structure exists but unfortunately, we do not know the values of the parameters. These parameters are held to exist but their true values are unknown. Hence there is a need to develop estimates of the true but unknown parameters. These estimates are based on a sample of data and it is to be hoped that they are in some sense acceptable, i.e. good.
The most common estimating approach is the principle of Ordinary Least Squares (OLS). As we have already learnt, this idea is easiest to understand when there is only one explanatory variable and the problem can be easily depicted with a scatter diagram onto which we can superimpose the line of best fit. This process of finding the line of best fit has been programmed in R through the lm()
function. Problems involving more than one explanatory variable are handled in a similar fashion. As part of the model fitting process, R produces a great deal of output which should be used to judge the validity, or usefulness, of the fitted model.
Model Validation
On deriving coefficient estimates from a sample of data we end up with a fitted regression model. We must now take this estimated model and ask a series of questions to decide whether or not our estimated model is good or bad. That is, we have to subject the fitted model to a set of tests designed to check the validity of the model, which is in effect a test of our viewpoint.
Test a): Does the fitted model make sense?
This type of testing is based on prior knowledge. We need to be in a position to judge if the parameter estimates are meaningful. This comes in two guises:
This test assesses if the marginal response coefficients display both the correct signs and sizes. To assist the analyst with these issues theoretical reasoning and information gained from past studies should be used. For example, if one was relating sales to advertising then a positive connection would be expected and a simple plot of sales against advertising would reveal information concerning the direction and strength of any sample relationship. Similarly, in a study of sales against price then a negative relationship would be expected. The issue of “correct size” is normally more troublesome. Possibly some clue as to size can be drawn from similar, past studies and this can be compared with the current estimated values.
By way of addressing this test the modeller may adopt the following approach:
Test b) \(R^2 / R^2_{adjusted}\): Overall is the model a good fit?
The coefficient of determination, \(R^2\), is a single number that measures the extent to which the set of explanatory variables can explain, or account for, the variability in \(Y\). That is, how well does the set of explanatory variables explain the behaviour of the phenomenon we are trying to understand? It is looking at the set of explanatory variables as a whole and is making no judgement about the contribution or importance of any individual explanatory variable. By construction, \(R^2\) is constrained to lie in the following range:
\[0\% <---------- \;\; R^2 \;\; ----------> 100\%\]
Intuitively the closer \(R^2\) is to \(100\%\) then the better the model is in the sense that the set of explanatory variables can explain a lot of the variability in \(Y\). Conversely, a value of \(R^2\) close to \(0\) implies a weak, i.e. poor model, in that the set of explanatory variables can only account for a limited amount of the behaviour of \(Y\). But what about non-extreme values of \(R^2\)?
The adequacy of \(R^2\) can be judged both formally and informally.
The formal test is given by the result:
and the appropriate test statistic is such that \(F_{calc} \sim F(k, (n-(k+1)))\), obtained using a data set of size \(n\) for a fitted model with \(k\) number of explanatory variables.
The mechanics of an F-test are as follows:
Two vital pieces of information are required for an F-test: \(F_{calc}\) and \(F_{crit}\).
\(F_{calc}\) is displayed within the R’s standard output. \(F_{crit}\) however is a figure derived from degrees of freedom elements and a designated level of significance. For hypothesis testing the \(5\%\) significance (\(\alpha = 5\%\)) level is commonly used. The degrees of freedom elements are
The all important decision making rule is: if \(F_{calc}\) is less than \(F_{crit}\) then the null hypothesis \(H_0\) is accepted; if \(F_{calc}\) is larger than \(F_{crit}\) then the alternative hypothesis \(H_1\) is accepted:
if \(F_{calc} < F_{crit} => H_0\)
if \(F_{calc} > F_{crit} => H_1\)
This formal test involves a rather weak alternative hypothesis, which says only that \(R^2\) is significantly bigger than \(0\). If \(H_1\) is accepted we will have to make a judgement, without the aid of any further formal test, about the usefulness of \(R^2\) and hence the model being studied.
\(R^2 Adjusted\)
\(R^2\) can also be used in comparing competing models. If two or more models have been put forward to explain the same response variable then a reasonable rule is to rank the explanatory power of the models in terms of their \(R^2\) values. Thus the model with the highest \(R^2\) value would be the best model.
The information gathered from the individual regressions carried out in Test a) can be used to give an initial judgement and ranking of the relative importance of the various explanatory variables.
However this rule should be used carefully. It is a valid rule when the competing models have the same number of explanatory variables. It is not valid when comparing models that have different numbers of explanatory variables. It is intuitively obvious that a model with more explanatory variables will have a better chance of explaining the \(Y\) variable than a model with fewer variables. Thus the highest \(R^2\) rule is biased in favour of those models with more explanatory variables even when some of these explanatory variables are not very useful. In an attempt to redress the imbalance when comparing models with different numbers of explanatory variables, a different version of \(R^2\) called \(R^2_{adjusted}\) has been developed as follows:
\[\bar{R}^2 = 1 - \frac{(n-1)}{(n-(k+1))}(1-R^2)\]
💡 Note that the mathematically adopted way of writing \(R^2_{adjusted}\) is \(\bar{R}^2\).
By examining \(\bar{R}^2\) it can be seen that when \(k\) goes up it is possible for the value of \(\bar{R}^2\) to fall. Thus this statistic gives a better way of comparing models with different values for \(k\). As before the rule for comparing models is straightforward: choose the model with the highest \(\bar{R}^2\).
Test c) t-tests: Individually, are the explanatory variables important?
Using \(R^2\) to judge the ‘goodness’ of the set of explanatory variables does not tell us anything about the importance of any one single explanatory variable. Just because a set of variables is important does not necessarily mean that each individual variable is contributing towards explaining the behaviour of \(Y\). What is needed is a test than enables us to check the validity of each variable one at a time. Such a test procedure is available as follows:
The appropriate \(H_1\) for any particular variable is crucially connected with the prior view as to the nature of the connection between any one proposed explanatory variable and the response variable.
The appropriate test statistic involves a t-test based on \((n-(k+1))\) degrees of freedom.
The critical region, denoted by \(H_1\), is crucially dependent upon the nature of the alternative hypothesis as put forward by the data analyst. \(t_{calc}\) is generated from the R output. \(t_{crit}\) is a combination of the Degrees of Freedom from Error and the level of test significance. \(t_{crit}\) is the benchmark helping to determine whether one accepts or rejects the null Hypothesis.
If \(t_{calc}\) lies in the critical region then the alternative hypothesis is accepted and the modeller accepts that the explanatory variable does have an influence on the behaviour of \(Y\). If this is not the case and \(t_{calc}\) lies in the \(H_0\) area, we will accept that the explanatory variable is useless and that it should be eliminated from the model.
The hypothesis testing methodology typically adopts the \(\alpha = 5\%\) level of significance and can be illustrated as follows:
Once all the variables have been individually analysed a further regression command should be given on all remaining acceptable explanatory variables in order to ascertain the best fitted model.
Some of the decision making outcomes from the series of t-test may be uncomfortable in terms of the information from other tests and we may have to experiment with various competing fitted models using subtle combinations of prior views, \(R^2\), \(\bar{R}^2\) values, and \(F\), \(t-test\) outcomes before selecting a best model or a set of equally good models.
There are many other, more sophisticated, tests that can be used to judge the validity, i.e. usefulness of a fitted regression model. Before we familiarise ourselves with them and in order to appreciate their practicality and effectiveness, we will conduct an analysis for fitting a multiple regression model based on the procedure explained above.
\(p\)-Value Approach for Hypothesis Testing
The \(p\)-value is a probability of obtaining a value of the test statistic or a more extreme value of the test statistic assuming that the null hypothesis is true. Thus, the \(p\)-value approach involves determining “likely” or “unlikely” by determining the probability, assuming the null hypothesis were true of observing a more extreme test statistic in the direction of the alternative hypothesis than the one observed.
As such, in the context of the inference about the regression parameters, the \(p\)-values help determine whether the relationships that you observe in your sample also exist in the larger population. The \(p\)-value for each independent variable tests the null hypothesis that the variable has no effect on the dependent variable. In other words, there is insufficient evidence to conclude that there is effect at the population level.
Case Study
carData::Salaries
: The 2008-09 nine-month academic salary for Assistant Professors, Associate Professors and Professors in a college in the U.S. The data was collected as part of the on-going effort of the college’s administration to monitor salary differences between male and female faculty members.
Format: A data frame with 397 observations on the following 6 variables:
rank
: a factor with levels:
discipline
: a factor with levels:
yrs.since.phd
: years since PhDyrs.service
: years of servicesex
: a factor with levels
salary
: nine-month salary, in dollarsThe first set of questions is:
Considering that the data is collected for the purpose of the analysis of academic salary we are going to identify salary
as the response variable. The rest of the variables we can use as possible factors that can influence the behaviour of the response variable.
The “Standard” regression approach assumes modelling a relationship between measured response and measured explanatory variables. However, often when building multiple regression models we do not want to be limited to the choice of only measured explanatory variables. We also want to be able to include attribute explanatory variables in the multiple regression modelling. In consequence, it is important to learn about supplementary steps that are required to make such models interpretable.
Dummy Variables
As we have learnt previously, attribute variables are vectors of class factor in R. This vector is encoded numerically, with information about the levels of the variable saved in the levels attribute.

However, when using an attribute variable in a linear regression model it would make no sense to treat it as a measured explanatory variable because of its numeric levels. In the context of linear modelling we need to code them to represent the levels of the attribute.
Two-level attribute variables are very easy to code. We simply create an indicator or dummy variable that takes on two possible dummy numerical values. Consider the sex
indicator variable. We can code this using a dummy variable \(d\):
\[\begin{equation}
d=\left\{
\begin{array}{@{}ll@{}}
0, & \text{if female} \\
1, & \text{if male}
\end{array}\right.
\end{equation}\]
{{% notice warning %}}
💡 This is the default coding used in R. Zero value is assigned to the level which is first alphabetically, unless it is changed by using the releveld()
function for example, or by specifying the levels of the factor variable specifically.
{{% /notice %}}
For a simple regression model of salary
versus sex
:
\[salary = b_0 + b_1sex + e,\]
this results in the model
\[\begin{equation} salary_i = b_0 + b_1sex_i + e_i=\left\{ \begin{array}{@{}ll@{}} b_0 + b_1 \times 1 + e_i = b_0 + b_1 + e_i, & \text{if the person is male} \\ b_0 + b_1 \times 0 + e_i = b_0 + e_i, & \text{if the person is female} \end{array}\right. \end{equation}\]
where \(b_0\) can be interpreted as the average \(\text{salary}\) for females, and \(b_0 + b_1\) as the average \(\text{salary}\) for males. The value of \(b_1\) represents the average difference in \(\text{salary}\) between females and males.
We can conclude that dealing with an attribute variable with two levels in a linear model is straightforward. In this case, a dummy variable indicates whether an observation has a particular characteristic: yes/no. We can observe it as a “switch” in a model, as this dummy variable can only assume the values 0 and 1, where 0 indicates the absence of the effect, and 1 indicates the presence. The values 0/1 can be seen as off/on.
The way in which R codes dummy variables is controlled by the contrast option:
options("contrasts")
## $contrasts
## unordered ordered
## "contr.treatment" "contr.poly"
The output points out the conversion of the factor into an appropriate set of contrasts. In particular, the first one: for unordered factors, and the second one: the ordered factors. The former one is applicable in our context.
To explicitly identify the coding of the factor, i.e. dummy variable used by R, we can use the contrasts()
function.
contrasts(sex)
## Male
## Female 0
## Male 1
contrasts(discipline)
## B
## A 0
## B 1
contrasts(rank)
## AssocProf Prof
## AsstProf 0 0
## AssocProf 1 0
## Prof 0 1
Note that applied “contr.treatment” conversion takes only the value 0 or 1 and that for an attribute variable with k levels it will create k-1 dummy variables. One can argue that the printout of the function could have been more informative by putting indexed letter d as a header for each of the printed columns. For example:
sex
, where k=2| attribute | \(d\) |
|---|---|
| Female | 0 |
| Male | 1 |
rank
, where k=3| attribute | \(d_1\) | \(d_2\) |
|---|---|---|
| AsstProf | 0 | 0 |
| AssocProf | 1 | 0 |
| Prof | 0 | 1 |
There are many different ways of coding attribute variables besides the dummy variable approach explained here. All of these different approaches lead to equivalent model fits. What differ are the coefficients, i.e. model parameters as they require different interpretations, arranged to measure particular contrasts. This 0/1 coding implemented in R’s default contr.treatment
contrast offers straightforward interpretation of the associated parameter in the model, which often is not the case when implementing other contrasts.
Interpreting coefficients of attribute variables
In the case of measured predictors, we are comfortable with the interpretation of the linear model coefficient as a slope, which tells us what a unit increase in the response variable is, i.e. outcome per unit increase in the explanatory variable. This is not necessarily the right interpretation for attribute explanatory variables.
# average salary values for each sex group
suppressPackageStartupMessages(library(dplyr))
Salaries %>%
select(salary, sex) %>%
group_by(sex) %>%
summarise(mean=mean(salary))
## # A tibble: 2 x 2
## sex mean
## <fct> <dbl>
## 1 Female 101002.
## 2 Male 115090.
If we obtain the mean salary for each sex group we will find that for female professors the average salary is \(\$101,002\) and for male professors the average is \(\$115,090\). That is, a difference of \(\$14,088\). If we now look at the parameters of the regression model for salary
vs sex
where females are coded as zero and males as one, we get exactly the same information, implying that the coefficient is the estimated difference in average between the two groups.
# regression model
lm(salary ~ sex)
##
## Call:
## lm(formula = salary ~ sex)
##
## Coefficients:
## (Intercept) sexMale
##<PHONE_NUMBER>8
For more on this topic check the following link: Categorical variables and interaction terms in linear regression
Fitting a Multivariate Regression Model
We are interested in the extent to which variation in the response variable salary
is associated with variation in the explanatory variables available in the Salaries
data set, that is we want to fit a multiple linear regression model to the given data. The model we wish to construct should contain enough to explain relations in the data and at the same time be simple enough to understand, explain to others, and use.
For convenience we will adopt the following notation:
salary
yrs.since.phd
yrs.service
discipline
sex
rank
Generally, in multiple regression we have a continuous response variable and two or more continuous explanatory variables, however in this dataset we have three attribute variables that we wish to include as the explanatory variables in the model.
Next, we need to specify the model that embodies our mechanistic understanding of the factors involved and the way that they are related to the response variable. It would make sense to expect that all of the available \(x\) variables may impact the behaviour of \(y\), thus the model we wish to build should reflect our viewpoint, i.e. \(y = f(x_1, x_2, x_3, x_4, x_5)\):
\[y = b_0 + b_1x_1 + b_2x_2 + b_3x_3 + b_4x_4 + b_5x_5 + e\] Our viewpoint states a belief that all explanatory variables have positive impact on the response. For example, more years in service will cause higher salary.
Our objective now is to determine the values of the parameters in the model that lead to the best fit of the model to the data. That is, we are not only trying to estimate the parameters of the model, but we are also seeking the minimal adequate model to describe the data.
{{% notice tip %}} The best model is the model that produces the least unexplained variation following the principle of parsimony rather than complexity. That is the model should have as few parameters as possible, subject to the constraint that the parameters in the model should all be statistically significant. {{% /notice %}}
For regression modelling in R we use the lm()
function, that fits a linear model assuming normal errors and constant variance. We specify the model by a formula that uses arithmetic operators: +
, -
, *
, /
and ^
which enable different functionalities from their ordinary ones. But, before we dive into statistical modelling of the given data, we need to take a first step and conduct the most fundamental task of data analysis procedure: Get to Know Our Data.
Examining multivariate data is intrinsically more challenging than examining univariate and bivariate data. To get the most in-depth vision into multivariate data behaviour we construct scatter plot matrices that enable the display of pairwise relationships. In R, the scatter plot matrices are composed by the pairs()
function, which comes as a part of the default graphics
package. Since we wish to include attribute variables in our analysis we are going to use the GGally::ggpairs()
function that produces a pairwise comparison of multivariate data for both data types: measured and attribute.
# If you don't have GGally installed yet, uncomment and run the line below
# install.packages(GGally)
suppressPackageStartupMessages(library(GGally))
ggpairs(Salaries)
This is an information rich visualisation that includes pairwise relationships of all the variables we want to consider for our model. By focusing on the last column of the plots, we can notice influence from all explanatory variables onto the response, except maybe for discipline
and sex
. We also notice unbalanced representation of the groups for variables rank
and sex
, but for the purpose of our practice in fitting a multi factor model this isn’t too problematic. We need to be especially concerned with the extent of correlations between the explanatory variables and what is of particular interest to us is the high multicollinearity between rank
, yrs.since.phd
and yrs.service
, which happens when the variables are highly linearly related.
As a consequence, we will need to keep an eye on the significance of using all of these variables in the model.
Fitting the Model
There are no fixed rules when fitting the linear models, but there are adopted standards that have proven to work well in practice. We start off by fitting a maximal model then we carry on simplifying it by removing non-significant explanatory variables. This needs to be done with caution, making sure that the simplifications make good scientific sense, and do not lead to significant reductions in explanatory power. Although this should be the adopted strategy for fitting a model it is not a guarantee to finding all the important structures in a complex data frame.
We can summarise our model building procedure algorithm as follows:
# model_1 <- lm(salary ~ yrs.since.phd + yrs.service + discipline + sex + rank, data = Salaries) #long handed way
model_1 <- lm(salary ~ ., data = Salaries) # full stop, . , implies: all other variables in data that do not already appear in the formula
summary(model_1)
##
## Call:
## lm(formula = salary ~ ., data = Salaries)
##
## Residuals:
## Min 1Q Median 3Q Max
## -65248 -13211 -1775 10384 99592
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 65955.2 4588.6 14.374 < 2e-16 ***
## rankAssocProf 12907.6 4145.3 3.114 0.00198 **
## rankProf 45066.0 4237.5 10.635 < 2e-16 ***
## disciplineB 14417.6 2342.9 6.154 1.88e-09 ***
## yrs.since.phd 535.1 241.0 2.220 0.02698 *
## yrs.service -489.5 211.9 -2.310 0.02143 *
## sexMale 4783.5 3858.7 1.240 0.21584
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 22540 on 390 degrees of freedom
## Multiple R-squared: 0.4547, Adjusted R-squared: 0.4463
## F-statistic: 54.2 on 6 and 390 DF, p-value: < 2.2e-16
The \(R^2 = 45.47\%\) and the \(\bar{R}^2 = 44.63\%\) are well above the value of zero allowing us to accept this as a valid model without having to formally test it to assess its statistical significance. It manages to explain almost half of the variability in the response variable salary
.
We identify the sex
explanatory variable as clearly not significant, which is in line with the conclusion we could draw from the boxplot in the pairwise comparison plot for salary
vs. sex
. We will remove it to begin the process of model simplification.
#model_1 <- lm(salary ~ yrs.since.phd + yrs.service + discipline + sex + rank, data = Salaries) # long handed method
model_2 <- update(model_1,~. - sex) # refitting by removing the least significant term
summary(model_2)
##
## Call:
## lm(formula = salary ~ rank + discipline + yrs.since.phd + yrs.service,
## data = Salaries)
##
## Residuals:
## Min 1Q Median 3Q Max
## -65244 -13498 -1455 9638 99682
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 69869.0 3332.1 20.968 < 2e-16 ***
## rankAssocProf 12831.5 4147.7 3.094 0.00212 **
## rankProf 45287.7 4236.7 10.689 < 2e-16 ***
## disciplineB 14505.2 2343.4 6.190 1.52e-09 ***
## yrs.since.phd 534.6 241.2 2.217 0.02720 *
## yrs.service -476.7 211.8 -2.250 0.02497 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 22550 on 391 degrees of freedom
## Multiple R-squared: 0.4525, Adjusted R-squared: 0.4455
## F-statistic: 64.64 on 5 and 391 DF, p-value: < 2.2e-16
We note a slight reduction in \(\bar{R^2}\) from \(44.63\%\) to \(44.55\%\) which we can regard as an insignificant decrease. The next step is to check the coefficients and assess for the effect of the remaining variables. We identify yrs.since.phd
and yrs.service
as the least influential in explaining the variability of salary
. To illustrate how to formally assess their effect, we will conduct the t-test for the yrs.since.phd
variable:
=========================
qt(0.95, 391)
## [1] 1.64876
As \(t_{calc} = 2.217 > t_{crit} = 1.64876 => H_1\) we will keep the remaining variable and stop with the model simplification and focus on its interpretation.
We can take a closer look at the coefficients of our fitted model:
coef(model_2)
## (Intercept) rankAssocProf rankProf disciplineB yrs.since.phd
## 69869.0110 12831.5375 45287.6890 14505.1514 534.6313
## yrs.service
## -476.7179
The structure of our final fitted model is:
\[y = b_0 + b_1x_1 + b_2x_2 + b_3x_3 + b_4x_4 + e,\] where
salary
yrs.since.phd
yrs.service
discipline
rank
We can take a closer look at the coefficients of our fitted model:
coef(model_2)
## (Intercept) rankAssocProf rankProf disciplineB yrs.since.phd
## 69869.0110 12831.5375 45287.6890 14505.1514 534.6313
## yrs.service
## -476.7179
Examining the output we realise that R
has created three “sub” dummy variables for the variable rank
:
\[
dr_1 =
\begin{cases}
1 & \text{rank is AsstProf} \\
0 & \text{for rank is not AsstProf}
\end{cases}
\]
\[ dr_2 = \begin{cases} 1 & \text{rank is AssocProf} \\ 0 & \text{rank is not AssocProf} \end{cases} \]
\[ dr_3 = \begin{cases} 1 & \text{rank is Prof} \\ 0 & \text{rank is not Prof} \end{cases} \]
It has chosen to use the model: \[ y = b_0 + b_1dr_2 + b_2dr_3 + b_3d_1 + b_4x_1 + b_5x_2 + e, \]
where:
salary
yrs.since.phd
yrs.service
rank
discipline
as explained earlierNote that R
doesn’t need to use \(dr_1\), to create three models; it only needs two dummy variables since it is using \(dr_1\) as a reference level, also known as the base line. This subsequently allows R
to create three models relating to rank
variable:
telling us that:
Learning this we can make an interpretation of our final fitted model as follows:
yrs.since.phd
) on average salary (salary
) will go up by \(\$534.63\) assuming the rest of the variables are fixed in the modelyrs.service
) on average salary (salary
) will go down by \(\$476.72\) assuming the rest of the variables are fixed in the modelrank: AsstProf
) who works in a “theoretical” department is \(\$69,869.01\) and who works in an “applied” department is \(\$84,374.16\); this can vary for the number of years in service and since PhDrank: AssocProf
) who works in a “theoretical” department is \(\$82,700.55\) and who works in an “applied” department is \(\$97,205.70\); this can vary for the number of years in service and since PhDrank: Prof
) who works in a “theoretical” department is \(\$115,156.70\) and who works in an “applied” department is \(\$129,661.90\); this can vary for the number of years in service and since PhDThis model explains around \(45\%\) of the variability in the response variable salary
.
Adding ~ 0
to lm()
formula enables R
to suppress the intercept. Note that if we remove the intercept, then we can directly obtain all “three intercepts” without a base level to fit the final fitted model:
model_2_1 <- lm(salary ~ 0 + rank + discipline + yrs.since.phd + yrs.service)
summary(model_2_1)
##
## Call:
## lm(formula = salary ~ 0 + rank + discipline + yrs.since.phd +
## yrs.service)
##
## Residuals:
## Min 1Q Median 3Q Max
## -65244 -13498 -1455 9638 99682
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## rankAsstProf 69869.0 3332.1 20.968 < 2e-16 ***
## rankAssocProf 82700.5 3916.7 21.115 < 2e-16 ***
## rankProf 115156.7 4350.9 26.467 < 2e-16 ***
## disciplineB 14505.2 2343.4 6.190 1.52e-09 ***
## yrs.since.phd 534.6 241.2 2.217 0.0272 *
## yrs.service -476.7 211.8 -2.250 0.0250 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 22550 on 391 degrees of freedom
## Multiple R-squared: 0.9638, Adjusted R-squared: 0.9633
## F-statistic: 1736 on 6 and 391 DF, p-value: < 2.2e-16
Adding a Complexity
So far we have seen that by fitting an additive regression model to the data, we aim to identify and understand how the value of a dependent response variable changes when any one of the independent explanatory variables is changed while the other independent variables stay the same. This is a restrictive form of a model as it only allows for linear relationships between the response and the explanatory variables, and the way in which one explanatory variable affects the response is the same for any value of the other explanatory variables used in the model.
We need to add flexibility to accommodate these limitations. This will allow the use of linear models for non-linear relationships and in which the effect of one explanatory variable can be different for different values of the other explanatory variable by introducing the concept of interaction. This brings more complexity into the multivariate regression model.
Case Study: The Quality of red Bordeaux Vintages
To illustrate the multivariate model fitting procedure with interactions we are going to use wine.csv
available from Eduardo García Portugués’s book: Notes for Predictive Modeling. The dataset is formed by the auction Price of 27 red Bordeaux vintages, five vintage descriptors (WinterRain, AGST, HarvestRain, Age, Year), and the population of France in the year of the vintage, FrancePop.
Year
: year in which grapes were harvested to make winePrice
: logarithm of the average market price for Bordeaux vintages according to 1990–1991 auctions. The price is relative to the price of the 1961 vintage, regarded as the best one ever recordedWinterRain
: winter rainfall (in mm)AGST
: Average Growing Season Temperature (in degrees Celsius)HarvestRain
: harvest rainfall (in mm)Age
: age of the wine measured as the number of years stored in a caskFrancePop
: population of France at Year (in thousands)We would like to analyse the quality of a vintage that has been quantified as the price and make the interpretation of our statistical findings.
{{% notice warning %}} Don’t forget!!! 🤔 First things first: Get to Know Data 🤓 {{% /notice %}}
We will start this analysis by examining the pairwise plot.
wine = read.csv("https://raw.githubusercontent.com/egarpor/handy/master/datasets/wine.csv")
summary(wine)
## Year Price WinterRain AGST HarvestRain
## Min. :1952 Min. :6.205 Min. :376.0 Min. :14.98 Min. : 38.0
## 1st Qu.:1960 1st Qu.:6.508 1st Qu.:543.5 1st Qu.:16.15 1st Qu.: 88.0
## Median :1967 Median :6.984 Median :600.0 Median :16.42 Median :123.0
## Mean :1967 Mean :7.042 Mean :608.4 Mean :16.48 Mean :144.8
## 3rd Qu.:1974 3rd Qu.:7.441 3rd Qu.:705.5 3rd Qu.:17.01 3rd Qu.:185.5
## Max. :1980 Max. :8.494 Max. :830.0 Max. :17.65 Max. :292.0
## Age FrancePop
## Min. : 3.00 Min. :43184
## 1st Qu.: 9.50 1st Qu.:46856
## Median :16.00 Median :50650
## Mean :16.19 Mean :50085
## 3rd Qu.:22.50 3rd Qu.:53511
## Max. :31.00 Max. :55110
ggpairs(wine)
What conclusions can we draw:
We can notice a perfect relationship between the variables Year
and Age
. This is to be expected since this data was collected in 1983 and Age
was calculated as: Age
= 1983 - Year
. Knowing this, we are going to remove Year
from the analysis and use Age
as it will be easier to interpret.
There is a strong relationship between Year
, ie. Age
and FrancePop
and since we want to impose our viewpoint that the total population does not influence the quality of wine we will not consider this variable in the model.
We are going to investigate possible interactions between the rainfall (WinterRain
) and growing season temperature (AGST
). In R
this will be created automatically by using the *
operator.
Let us build a model:
We will start with the most complicated model that includes the highest-order interaction.
{{% notice note %}} In R we will specify the three-way interaction, which will automatically add all combinations of two-way interactions. {{% /notice %}}
model1 <- lm(Price ~ WinterRain + AGST + HarvestRain + Age + WinterRain * AGST * HarvestRain, data = wine)
summary(model1)
##
## Call:
## lm(formula = Price ~ WinterRain + AGST + HarvestRain + Age +
## WinterRain * AGST * HarvestRain, data = wine)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.35058 -0.19462 -0.02645 0.17194 0.52079
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 8.582e+00 1.924e+01 0.446 0.6609
## WinterRain -1.858e-02 2.896e-02 -0.642 0.5292
## AGST -1.748e-01 1.137e+00 -0.154 0.8795
## HarvestRain -4.713e-02 1.540e-01 -0.306 0.7631
## Age 2.476e-02 8.288e-03 2.987 0.0079 **
## WinterRain:AGST 1.272e-03 1.712e-03 0.743 0.4671
## WinterRain:HarvestRain 7.836e-05 2.600e-04 0.301 0.7665
## AGST:HarvestRain 3.059e-03 9.079e-03 0.337 0.7401
## WinterRain:AGST:HarvestRain -5.446e-06 1.540e-05 -0.354 0.7278
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.2833 on 18 degrees of freedom
## Multiple R-squared: 0.8621, Adjusted R-squared: 0.8007
## F-statistic: 14.06 on 8 and 18 DF, p-value: 2.675e-06
The model explains well over \(80\%\) of variability and is clearly a strong model, but the key question is whether we can simplify it. We will start the process of this model simplification by removing the three-way interaction as it is clearly not significant.
model2 <- update(model1, ~. -WinterRain:AGST:HarvestRain, data =wine)
summary(model2)
##
## Call:
## lm(formula = Price ~ WinterRain + AGST + HarvestRain + Age +
## WinterRain:AGST + WinterRain:HarvestRain + AGST:HarvestRain,
## data = wine)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.35245 -0.19452 0.01643 0.17289 0.51420
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 2.980e+00 1.066e+01 0.279 0.78293
## WinterRain -9.699e-03 1.408e-02 -0.689 0.49930
## AGST 1.542e-01 6.383e-01 0.242 0.81168
## HarvestRain 6.496e-03 2.610e-02 0.249 0.80610
## Age 2.441e-02 8.037e-03 3.037 0.00678 **
## WinterRain:AGST 7.490e-04 8.420e-04 0.890 0.38484
## WinterRain:HarvestRain -1.350e-05 7.338e-06 -1.840 0.08144 .
## AGST:HarvestRain -1.032e-04 1.520e-03 -0.068 0.94656
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.2767 on 19 degrees of freedom
## Multiple R-squared: 0.8611, Adjusted R-squared: 0.8099
## F-statistic: 16.83 on 7 and 19 DF, p-value: 6.523e-07
The \(\bar{R}^2\) has slightly increased in value. Next, we remove the least significant two-way interaction term.
model3 <- update(model2, ~. -AGST:HarvestRain, data = wine)
summary(model3)
##
## Call:
## lm(formula = Price ~ WinterRain + AGST + HarvestRain + Age +
## WinterRain:AGST + WinterRain:HarvestRain, data = wine)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.35424 -0.19343 0.01176 0.17161 0.51218
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 3.518e+00 6.946e+00 0.507 0.61802
## WinterRain -1.017e-02 1.195e-02 -0.851 0.40488
## AGST 1.218e-01 4.138e-01 0.294 0.77147
## HarvestRain 4.752e-03 4.553e-03 1.044 0.30901
## Age 2.451e-02 7.710e-03 3.179 0.00472 **
## WinterRain:AGST 7.769e-04 7.166e-04 1.084 0.29119
## WinterRain:HarvestRain -1.342e-05 7.059e-06 -1.902 0.07174 .
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.2697 on 20 degrees of freedom
## Multiple R-squared: 0.8611, Adjusted R-squared: 0.8194
## F-statistic: 20.66 on 6 and 20 DF, p-value: 1.35e-07
Again, it is reassuring to notice an increase in the \(\bar{R}^2\), but we can still simplify the model further by removing another least significant two-way interaction term.
model4 <- update(model3, ~. -WinterRain:AGST, data = wine)
summary(model4)
##
## Call:
## lm(formula = Price ~ WinterRain + AGST + HarvestRain + Age +
## WinterRain:HarvestRain, data = wine)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.5032 -0.1934 0.0109 0.1771 0.4621
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -3.812e+00 1.598e+00 -2.386 0.026553 *
## WinterRain 2.747e-03 9.471e-04 2.900 0.008560 **
## AGST 5.586e-01 9.495e-02 5.883 7.71e-06 ***
## HarvestRain 4.717e-03 4.572e-03 1.032 0.313877
## Age 2.785e-02 7.094e-03 3.926 0.000774 ***
## WinterRain:HarvestRain -1.349e-05 7.088e-06 -1.903 0.070835 .
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.2708 on 21 degrees of freedom
## Multiple R-squared: 0.8529, Adjusted R-squared: 0.8179
## F-statistic: 24.35 on 5 and 21 DF, p-value: 4.438e-08
There is an insignificant decrease in \(\bar{R}^2\). We notice HarvestRain
is now the least significant term, but it is used for the WinterRain:HarvestRain
interaction, which is significant at \(\alpha = 5\%\) and therefore we should keep it. However, as the concept of parsimony prefers a model without interactions to a model containing interactions between variables, we will remove the remaining interaction term and see if it significantly affects the explanatory power of the model.
model5 <- update(model4, ~. -WinterRain:HarvestRain, data = wine)
summary(model5)
##
## Call:
## lm(formula = Price ~ WinterRain + AGST + HarvestRain + Age, data = wine)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.46024 -0.23862 0.01347 0.18601 0.53443
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -3.6515703 1.6880876 -2.163 0.04167 *
## WinterRain 0.0011667 0.0004820 2.420 0.02421 *
## AGST 0.6163916 0.0951747 6.476 1.63e-06 ***
## HarvestRain -0.0038606 0.0008075 -4.781 8.97e-05 ***
## Age 0.0238480 0.0071667 3.328 0.00305 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.2865 on 22 degrees of freedom
## Multiple R-squared: 0.8275, Adjusted R-squared: 0.7962
## F-statistic: 26.39 on 4 and 22 DF, p-value: 4.057e-08
The \(\bar{R}^2\) is reduced by around \(2\%\), but it has all the significant terms and it is easier to interpret. For those reasons and in the spirit of parsimony that argues that a model should be as simple as possible, we can suggest that this should be regarded as the best final fitted model.
We realise that for the large numbers of explanatory variables, and many interactions and non-linear terms, the process of model simplification can take a very long time. There are many algorithms for automatic variable selection that can help us to chose the variables to include in a regression model. Stepwise regression and Best Subsets regression are two of the more common variable selection methods.
The stepwise procedure starts from the saturated model (or the maximal model, whichever is appropriate) through a series of simplifications to the minimal adequate model. This progression is made on the basis of deletion tests: F tests, AIC, t-tests or chi-squared tests that assess the significance of the increase in deviance that results when a given term is removed from the current model.
The best subset regression (BREG), also known as “all possible regressions”, as the name of the procedure indicates, fits a separate least squares regression for each possible combination of the \(p\) predictors, i.e. explanatory variables. After fitting all of the models, BREG then displays the best fitted models with one explanatory variable, two explanatory variables, three explanatory variables, and so on. Usually, either adjusted R-squared or Mallows Cp is the criterion for picking the best fitting models for this process. The result is a display of the best fitted models of different sizes up to the full/maximal model and the final fitted model can be selected by comparing displayed models based on the criteria of parsimony.
“These methods are frequently are abused by naive researchers who seek to interpret the order of entry of variables into the regression equation as an index of their ‘‘importance.’’ This practice is potentially misleading.” J. Fox and S. Weisberg book: An R Companion to Applied Regression, Third Edition, Sage (2019)
{{% notice tip %}} 💡 When selecting a model one should remember the important concept of parsimony! {{% /notice %}}
As M.J. Crawley points out in his well know editions of “The R Book” we need to remember that models are portrayals of phenomena that should be both “accurate and convenient” and the principle of parsimony is an essential tool for model exploration. As he suggests: “just because we go to the trouble of measuring something does not mean we have to have it in our model.”
"Parsimony says that, other things being equal, we prefer:
Useful links:
Best Subset, https://afit-r.github.io/model_selection, https://bookdown.org/tpinto_home/Regularisation/best-subset-selection.html
Summary
For problems ranging from bioinformatics to marketing, many analysts prefer to develop “classifiers” instead of developing predictive models
Further Reading
Claeskens, G. and Hjort, N.L. (2008) Model Selection and Model Averaging, Cambridge University Press, Cambridge.
Fox, J. (2002) An R and S-Plus Companion to Applied Regression, Sage, Thousand Oaks, CA.
YOUR TURN 👇
Salaries
data Case Study:~0
to lm()
formula enables R
to suppress the intercept. Try to fit the following model by removing the intercept.model_2_1 <- lm(salary ~ 0 + rank + discipline + yrs.since.phd + yrs.service)
summary(model_2_1)
This will allow you to obtain all “three intercepts” without a reference level.
Does this model differ from the previously fitted model_2
? Provide justified explanation
Interpret the model
Salaries
data Case Study be further simplified? Justify your answerPrestige
data available from the carData
package of datasets designed to accompany J. Fox and S. Weisberg book: An R Companion to Applied Regression, Third Edition, Sage (2019). Fit a multivariate model that explains variation in the response variable prestige
, explaining the reasons behind the steps taken with appropriate interpretation of the final fitted model.carData::Prestige
: The Canadian occupational prestige data, where the observations are occupations. Justification for treating the 102 occupations as a sample implicitly rests on the claim that they are “typical” of the population, at least with respect to the relationship between prestige and income.
education
: Average education of occupational incumbents, years, in 1971.
income
: Average income of incumbents, dollars, in 1971.
women
: Percentage of incumbents who are women.
prestige
: Pineo-Porter prestige score for occupation, from a social survey conducted in the mid-1960s.
census
: Canadian Census occupational code.
type
: Type of occupation. A factor with levels (note: out of order):
bc
: Blue Collarprof
: Professional, Managerial, and Technicalwc
: White CollarProstate
available in the package lasso2
contains information on 97 men who were about to receive a radical prostatectomy. This data set comes from a study examining the correlation between the prostate specific antigen (lpsa
) and a number of other clinical measures.lpsa
using all the available predictors.Take a look at vignette for the lasso2
package
© 2021 Tatjana Kecojevic


OH NO!
You made the wrong choice and drifted off into space, never to be heard from again.
Just kidding!
Try again, maybe?
You made the wrong choice and drifted off into space, never to be heard from again.
Just kidding!
Try again, maybe?


CHAPTER 323B
HEALTH CARE PRIVACY HARMONIZATION ACT
Section
323B-1 Applicability
323B-2 Definitions
323B-3 Privacy of individually identifiable health
information
323B-4 Relationship to other laws
323B-5 Disclosure, use, and production of de-identified
protected health information


This calculator will help you converts a spectral bandwidth expressed in wavelength Δλ at a certain central wavelength λ0to frequency domain.
The formula that will be used is as followed:
Wavelength Bandwidth $\lambda$; Center Wavelength $\lambda_0$;
Center Frequency $V_0$; Frequency Bandwidith $\triangle V$;


by Ryan Rose(C) June 2016
Please input a Facebook Page id. To find an id for a known page please use findmyfbid.com. (Ex.<PHONE_NUMBER>0,<PHONE_NUMBER>7)
Please enter a date range of equal or less than five days, then select a post.
Please select a post.
Generate Composition


--- layout: post title: Phonemic Awareness and Reading Readiness date: 2009-11-22 09:21:38.000000000 -08:00 type: post parent_id: '0' published: true pd: '' status: publish categories: - Early Childhood Issues tags: - learning games - phonemic awareness - phonics - reading readiness author: login: k email: k@k display_name: mcary first_name: '' last_name: '' ---
I am looking for some help. Recently I attended an Open House at my daughter’s school. The kindergarten teacher mentioned the skills that children would be working on during the year. She didn’t get into specifics in each area. One thing that she mentioned was, “phonemic awareness.” I am not sure what that is or how I can help my daughter. If you have any suggestions, I would appreciate the help.
To put it simply, phonemic awareness is the ability to hear and manipulate sounds (or phonemes) in spoken language as they relate to written language. It is not to be confused with phonics which is a code to sound out written words. I like to describe phonemic awareness in young children as their ability to hear, use, and understand the rhyme and rhythm in our speech.
Phonemic awareness improves children’s ability to read, spell and comprehend what is read. I’ve noticed a decline in the phonemic awareness of many children entering school over the past few years. One reason may be that many parents have gotten away from reading nursery rhymes to their children. As a result, children often come to school not knowing any rhymes at all. Repetitive poems and simple rhymes such as those found in Mother Goose Nursery Rhymes are helpful in teaching phonemic awareness.
Parents should start teaching phonemic awareness when their child is a toddler so that they will have the pre-reading skills they need when they begin school. Practicing phonemic awareness should be a fun and easy activity. Here are a few simple games that parents can play with their child whenever they might have a few free minutes. As most activities do not require a paper and pencil, you can practice (play) them anywhere, even in the car.
PHONEMIC AWARENESS activities include:
1) RHYMING
Recognizing rhymes in books and poems - Read a rhyming book or poem and have the child pick out the rhyming words at the end of each line.
Saying rhymes - Example: “Which two words rhyme: The silly fox sat on the box?”
Recognizing rhymes - Example: “Do these words rhyme: down/dip; down/clown?”
Generating rhymes - Example: “Can you think of a word that rhymes with frog?”
As many other games in this piece, the above are great car games that the whole family can join in on.
2) SEGMENTING SYLLABLES
Say aloud and clap the syllables in words - Example: “How many syllables are in the words: cat; cupcake; elephant; television; etc.?
3) BEGINNING SOUND SUBSTITUTION
Change the first sound of the word to make a new word - Examples:
Ask: “What would I do to change the word cat to mat?”
“What would I do to change the word cook to shook [harder]?”
“Can you think of a word that rhymes with sock, but starts with an /l/?”
4) TEACHING SOUNDS IN ISOLATION
Identify individual sounds in words - Examples:
“What sound do you hear at the beginning of the word hen?”
“What sound do you hear in the middle if the word pig?”
“What is the ending sound of lion?”
“What sound do all of these words begin with: duck; dog; damp?”
“What do these words end with: sock; quack; kick?”
Identify words from individual sounds - Examples:
Slowly stretch out the sounds in the word: c / a / t. Then ask, “What word am I saying?”
Slowly stretch out the sounds in the word: j / u / m / p. Then ask, “What word am I saying?”
When your daughter has mastered easier concepts, make your games a little harder, move onto higher level skills. Examples: “Can you say cat without the /C/ ?’ “Can you say brook without the /Br/ ?
By practicing these simple activities with your daughter, you will be helping her master phonemic awareness, an important skill in learning to read!


MESSAGES FROM ANOTHER PLACE
Let's Rock
P98/V367/J876/J87/K98/G34/647/Y78/DF45/J567
8765/ER23/PI36/K876/B256/J876/JK26/K89/Y567
T456/K87/U78/H54/M45/D23/M66/J86/BS28/DF321
JK78/F45/U78/R348/J57/N454/K876/J76/M34/D23
SHARE/ON/TWITTER//VISIT/GITHUB/TO/KNOW/MORE
PI36/K78/B676/G565/C454/N669/H6776/M38/Y567
K57/THE/OWLS/ARE/NOT/WHAT/THEY/SEEM/N56/B56
554/B789/W26/MN78/N78/H45/JK78/DF45/M678/J3
It looks like Jupiter and Saturn haven't met yet... Or maybe you're just using a web browser that does not support audio recording.
Write it in your own diary.


T
- the type of data held in the boxpublic interface MarketDataBox<T>
A box can contain a single value for the data or it can contain multiple values, one for each scenario. If it contains a single value then the same value is used in every scenario.
Wrapping the data in a box allows a simple interface for looking up market data that hides whether there is one value or multiple values. Without the box every function that uses market data would have to handle the two cases separately.
The box also takes care of transforming the market data when using it to build other market data values
(see the apply
methods). This means that market data functions and perturbations don't need
different logic to deal with single and multiple values.
Using a box allows scenario data to be stored more efficiently in some cases. For example, curve data for multiple scenarios can include one copy of the x-axis data which is used in all scenarios. If a separate curve were stored for each scenario that data would be unnecessarily stored multiple times.
In some cases a function might need to access the data for all scenarios at the same time. For example, if
part of the calculation is the same for all scenarios it can be done once and reused instead of recalculated
for each scenario. In this case a ScenarioMarketDataId
should be used to retrieve the scenario
value from the market data container.
| Modifier and Type | Method and Description |
|---|---|
<U,R> MarketDataBox<R> |
combineWith(MarketDataBox<U> other,
BiFunction<T,U,R> fn)
Applies a function to the market data in this box and another box and returns a box containing the result.
|
static <T> MarketDataBox<T> |
empty()
Obtains an instance containing no market data.
|
Class<?> |
getMarketDataType()
Gets the type of the market data value used in each scenario.
|
int |
getScenarioCount()
Gets the number of scenarios for which this box contains data.
|
ScenarioArray<T> |
getScenarioValue()
Gets the market data value containing data for multiple scenarios.
|
T |
getSingleValue()
Gets the single market data value used for all scenarios if available.
|
T |
getValue(int scenarioIndex)
Gets the market data value associated with the specified scenario.
|
default boolean |
isScenarioValue()
Checks if this box contains market data for multiple scenarios.
|
boolean |
isSingleValue()
Checks if this box contains a single market data value that is used for all scenarios.
|
<R> MarketDataBox<R> |
map(Function<T,R> fn)
Applies a function to the contents of the box and returns another box.
|
<R> MarketDataBox<R> |
mapWithIndex(int scenarioCount,
ObjIntFunction<T,R> fn)
Applies a function to the contents of the box once for each scenario and returns a box containing
the values returned from the function.
|
static <T> MarketDataBox<T> |
ofScenarioValue(ScenarioArray<T> scenarioValue)
Obtains an instance containing a scenario market data value with data for multiple scenarios.
|
static <T> MarketDataBox<T> |
ofScenarioValues(List<T> scenarioValues)
Obtains an instance containing a scenario market data value with data for multiple scenarios.
|
static <T> MarketDataBox<T> |
ofScenarioValues(T... scenarioValues)
Obtains an instance containing a scenario market data value with data for multiple scenarios.
|
static <T> MarketDataBox<T> |
ofSingleValue(T singleValue)
Obtains an instance containing a single market data value that is used in all scenarios.
|
Stream<T> |
stream()
Returns a stream over the contents of the box.
|
static <T> Collector<T,List<T>,MarketDataBox<T>> |
toMarketDataBox()
Returns a collector which can be used to collect values from a stream into a market data box.
|
static <T> MarketDataBox<T> ofSingleValue(T singleValue)
T
- the type of the market data value used in each scenariosingleValue
- the market data value containing data for a single scenariostatic <T> MarketDataBox<T> ofScenarioValue(ScenarioArray<T> scenarioValue)
T
- the type of the market data value used in each scenarioscenarioValue
- the market data value containing data for multiple scenarios@SafeVarargs static <T> MarketDataBox<T> ofScenarioValues(T... scenarioValues)
The market data is made up of multiple single values, one for each scenario.
T
- the type of the market data value used in each scenarioscenarioValues
- the market data values for each scenariostatic <T> MarketDataBox<T> ofScenarioValues(List<T> scenarioValues)
T
- the type of the market data value used in each scenarioscenarioValues
- the market data values for each scenariostatic <T> MarketDataBox<T> empty()
T
- the type of the market data value used in each scenarioT getSingleValue()
If this box contains data for multiple scenarios an exception is thrown.
This method should only be called if isSingleValue()
returns true
or isScenarioValue()
return false
.
UnsupportedOperationException
- if this box contains data for multiple scenariosScenarioArray<T> getScenarioValue()
If this box contains data for a single scenario an exception is thrown.
This method should only be called if isSingleValue()
returns false
or isScenarioValue()
return true
.
UnsupportedOperationException
- if this box contains data for a single scenarioT getValue(int scenarioIndex)
scenarioIndex
- the index of the scenarioboolean isSingleValue()
default boolean isScenarioValue()
int getScenarioCount()
If a box contains data for a single scenario it is treated as a special case and can be used in any number of scenarios, returning the same value for every scenario.
Class<?> getMarketDataType()
<R> MarketDataBox<R> map(Function<T,R> fn)
The box implementation takes care of checking whether it contains a single value or a scenario value, applying the function to the value for each scenario and packing the return value into a box.
This is primarily intended for use by market data factories which might receive single values or scenario values from upstream market data factories.
R
- the return type of the functionfn
- a function to apply to the market data in the box<R> MarketDataBox<R> mapWithIndex(int scenarioCount, ObjIntFunction<T,R> fn)
The scenario count
of the box must be one or it must be equal to the
scenarioCount
argument. The scenario count
of the return value
will be equal to scenarioCount
.
The box implementation takes care of checking whether it contains a single value or a scenario value, applying the function to the value for each scenario and packing the return values into a box.
This is primarily intended to be used by perturbations which generate separate market data values for each scenario data by applying a function to the existing value for the scenario.
R
- the type of the returned market datascenarioCount
- the total number of scenariosfn
- the function that is invoked with a scenario index and the market data value for that scenario.
The return value is used as the scenario data in the returned box<U,R> MarketDataBox<R> combineWith(MarketDataBox<U> other, BiFunction<T,U,R> fn)
The box implementation takes care of checking whether the input boxes contain single values or a scenario values, applying the function to the value for each scenario and packing the return value into a box.
This is primarily intended for use by market data factories which might receive single values or scenario values from upstream market data factories.
U
- the type of market data in the other boxR
- the type of the market data returned in the result of the functionother
- another market data boxfn
- the function invoked with the market data from each box. The return value is used to build the data
in the returned boxStream<T> stream()
static <T> Collector<T,List<T>,MarketDataBox<T>> toMarketDataBox()
T
- the type of the items in the streamCopyright 2009-Present by OpenGamma Inc. and individual contributors
Apache v2 licensed
Additional documentation can be found at opengamma.github.io/StrataDocs.


Home
About
Services
Contact
Feedback
FERRARI WORLD
Ferrai
Price = $300
Book now
Ferrai
Price = $800
Book now
Ferrai
Price = $200
Book now
Ferrai
Price = $300
Book now
Ferrai
Price = $300
Book now
Ferrai
Price = $300
Book now


All you can eat pancakes served with our famous maple syrup on the side.
Optional toppings include: strawberries, mixed berries, and whipped cream. Additional .50¢
Poached eggs on top of maple bacon and a toasted english muffin, with hollandaise sauce drizzled on top.
Fried egg, American cheese, and maple bacon between a perfectly toasted English muffin. Comes with a side of fruit, oatmeal, or yogurt.
Fill our mouth-watering crepes with either cherries, strawberries, mixedberries, or cinnamon apples. Comes with a side of fruit, oatmeal, or yogurt.
Mix and match two eggs any way you want them with toast and two other sides.
Side choices: hashbrowns, home fries, bacon, sausage links, fruit, yogurt, oatmeal.
For those of you with a big morning appetite, order two pancakes or french toast alongside two eggs, toast, and two other sides of your choice.
Side choices: hashbrowns, home fries, bacon, sausage, fruit, yogurt, oatmeal.
A juicy burger cooked to your liking, topped with our cripsy maple bacon, lettuce, cheese, barbecue sauce. Sides include a crisp dill pickle and a side of fries.
A basket of five crispy chicken fingers and a side of fries. Your sauce choices include: barbecue, honey mustard, and buffalo.
Classic caesar salad with your choice of cripsy or grilled chicken in a bed of romaine lettuce, lemon juice, croutons, and parmesan cheese
A Wilson house favorite, this five cheese Mac and Cheese includes: cheddar, mozzarella, feta, gouda, and provalone topped with bread crumbs and baked to gooey perfection.
Enjoy our classic panini of sliced chicken breast, tomatoes, red onions, romaine lettuce, chipotle mayo on ciabatta bread. This comes with your choice of house soup or salad.
Perfect choice for kids or winding down from a long day. You may choose between mozzarella or American cheese.
Warm apple cinnamon slices in a buttery, brown sugar, cinnamon, ginger, and nutmeg glaze with an oats crumble. Top it all off with a scoop of vanilla ice cream.
Dig into a fluffy, chocolate brownie and release its oozing cholate center. This dessert is topped with powered sugar and a scoop of vanilla ice cream.
A sugar cookie crust topped with the rippest of mixed berries.
Free refills for all products after first purchase.


Box Office Records > Domestic > All Movies > Cumulative > All Time Domestic
All Time Domestic Box Office (Rank 1,301-1,400)
See also: All Time Domestic Limited Release - All Time International - All Time Worldwide - All Time Domestic (Inf. Adj.) - Top 2016 Domestic - Top 2017 Domestic
This chart contains the top 100 movies based on the cumulative domestic box office.
Please see our All-Time Top Grossing Films Comparison for a comparison of the daily performance of all films released since 1980 that have earned more than $450 million at the domestic box office.
| Rank | Released | Movie | Domestic Box Office |
International Box Office |
Worldwide Box Office |
|---|---|---|---|---|---|
| 1,301 | 2003 | Gothika | $59,588,068 | $81,896,744 | $141,484,812 |
| 1,302 | 1985 | Spies Like Us | $59,526,202 | $0 | $59,526,202 |
| 1,303 | 1991 | The Last Boy Scout | $59,509,925 | $0 | $59,509,925 |
| 1,304 | 2004 | The Stepford Wives | $59,475,623 | $36,746,348 | $96,221,971 |
| 1,305 | 2003 | Love Actually | $59,472,278 | $188,477,969 | $247,950,247 |
| 1,306 | 1988 | Scrooged | $59,450,353 | $0 | $59,450,353 |
| 1,307 | 2014 | Planes: Fire and Rescue | $59,157,732 | $97,241,912 | $156,399,644 |
| 1,308 | 2014 | The Grand Budapest Hotel | $59,076,019 | $105,274,390 | $164,350,409 |
| 1,309 | 1996 | Courage Under Fire | $59,003,384 | $41,829,761 | $100,833,145 |
| 1,310 | 2004 | Dawn of the Dead | $58,990,765 | $44,462,110 | $103,452,875 |
| 1,311 | 2015 | Vacation | $58,884,188 | $41,771,704 | $100,655,892 |
| 1,312 | 1992 | Far and Away | $58,883,840 | $0 | $58,883,840 |
| 1,313 | 2012 | Total Recall | $58,877,969 | $152,978,119 | $211,856,088 |
| 1,314 | 2007 | Surf's Up | $58,867,694 | $86,528,051 | $145,395,745 |
| 1,315 | 2001 | Behind Enemy Lines | $58,855,732 | $0 | $58,855,732 |
| 1,316 | 2009 | Bride Wars | $58,715,510 | $56,434,914 | $115,150,424 |
| 1,317 | 2011 | Water for Elephants | $58,709,717 | $58,100,000 | $116,809,717 |
| 1,318 | 2016 | Jack Reacher: Never Go Back | $58,697,076 | $101,341,331 | $160,038,407 |
| 1,319 | 1992 | Honey I Blew Up the Kid | $58,662,452 | $0 | $58,662,452 |
| 1,320 | 2006 | Curious George | $58,640,119 | $12,412,485 | $71,052,604 |
| 1,321 | 2006 | Little Man | $58,636,047 | $43,000,000 | $101,636,047 |
| 1,322 | 1996 | Jack | $58,617,334 | $0 | $58,617,334 |
| 1,323 | 2014 | RoboCop | $58,607,007 | $184,374,792 | $242,981,799 |
| 1,324 | 1989 | Sea of Love | $58,571,513 | $0 | $58,571,513 |
| 1,325 | 1992 | HouseSitter | $58,500,635 | $36,400,000 | $94,900,635 |
| 1,326 | 1968 | Funny Girl | $58,500,000 | $0 | $58,500,000 |
| 1,327 | 1992 | Death Becomes Her | $58,422,650 | $90,600,000 | $149,022,650 |
| 1,328 | 1997 | Anastasia | $58,403,409 | $81,397,881 | $139,801,290 |
| 1,329 | 2010 | Easy A | $58,401,464 | $17,799,257 | $76,200,721 |
| 1,330 | 2003 | Open Range | $58,331,254 | $10,282,738 | $68,613,992 |
| 1,331 | 2007 | Halloween | $58,269,151 | $19,245,250 | $77,514,401 |
| 1,332 | 2013 | The Secret Life of Walter Mitty | $58,236,838 | $130,300,000 | $188,536,838 |
| 1,333 | 1999 | Bicentennial Man | $58,220,776 | $29,200,000 | $87,420,776 |
| 1,334 | 2001 | Ali | $58,183,966 | $29,500,000 | $87,683,966 |
| 1,335 | 2004 | Without a Paddle | $58,156,435 | $6,964,845 | $65,121,280 |
| 1,336 | 1993 | Demolition Man | $58,055,768 | $101,000,000 | $159,055,768 |
| 1,337 | 2017 | Baywatch | $58,035,798 | $111,700,000 | $169,735,798 |
| 1,338 | 2008 | Step Up 2 the Streets | $58,017,783 | $90,569,127 | $148,586,910 |
| 1,339 | 1991 | My Girl | $58,011,485 | $0 | $58,011,485 |
| 1,340 | 2011 | The Lincoln Lawyer | $58,009,200 | $29,135,886 | $87,145,086 |
| 1,341 | 2004 | The Butterfly Effect | $57,924,679 | $37,713,913 | $95,638,592 |
| 1,342 | 2005 | Red Eye | $57,891,803 | $38,774,683 | $96,666,486 |
| 1,343 | 2004 | Shall We Dance? | $57,887,882 | $60,210,000 | $118,097,882 |
| 1,344 | 1998 | U.S. Marshals | $57,833,603 | $0 | $57,833,603 |
| 1,345 | 2006 | Invincible | $57,806,952 | $694,175 | $58,501,127 |
| 1,346 | 2010 | Takers | $57,744,720 | $12,842,548 | $70,587,268 |
| 1,347 | 2004 | The Chronicles of Riddick | $57,712,751 | $49,500,000 | $107,212,751 |
| 1,348 | 2016 | Fences | $57,682,904 | $3,881,620 | $61,564,524 |
| 1,349 | 1998 | Life is Beautiful | $57,598,247 | $171,787,114 | $229,385,361 |
| 1,350 | 1999 | The Cider House Rules | $57,547,209 | $31,000,000 | $88,547,209 |
| 1,351 | 1981 | The Cannonball Run | $57,498,710 | $0 | $57,498,710 |
| 1,352 | 1989 | Pet Sematary | $57,469,179 | $0 | $57,469,179 |
| 1,353 | 1987 | Dragnet | $57,387,000 | $0 | $57,387,000 |
| 1,354 | 2017 | The Shack | $57,386,418 | $35,556,407 | $92,942,825 |
| 1,355 | 2001 | Moulin Rouge | $57,386,369 | $121,823,316 | $179,209,685 |
| 1,356 | 2006 | Blood Diamond | $57,377,916 | $114,000,000 | $171,377,916 |
| 1,357 | 1994 | The Specialist | $57,362,581 | $0 | $57,362,581 |
| 1,358 | 1993 | Sister Act 2 Back in the Habit | $57,319,029 | $0 | $57,319,029 |
| 1,359 | 1974 | The Godfather: Part II | $57,300,000 | $0 | $57,300,000 |
| 1,360 | 1988 | Willow | $57,269,863 | $0 | $57,269,863 |
| 1,361 | 2000 | The Whole Nine Yards | $57,262,492 | $28,265,375 | $85,527,867 |
| 1,362 | 1980 | The Blues Brothers | $57,229,890 | $0 | $57,229,890 |
| 1,363 | 2000 | Next Friday | $57,176,582 | $2,498,725 | $59,675,307 |
| 1,364 | 1981 | Chariots of Fire | $57,159,946 | $307,785 | $57,467,731 |
| 1,365 | 1972 | What's Up, Doc? | $57,142,740 | $0 | $57,142,740 |
| 1,366 | 1995 | Twelve Monkeys | $57,141,459 | $111,700,000 | $168,841,459 |
| 1,367 | 2004 | 13 Going On 30 | $57,139,723 | $39,300,000 | $96,439,723 |
| 1,368 | 1992 | Unlawful Entry | $57,138,719 | $0 | $57,138,719 |
| 1,369 | 1982 | Annie | $57,059,003 | $0 | $57,059,003 |
| 1,370 | 1988 | Beaches | $57,041,866 | $0 | $57,041,866 |
| 1,371 | 2015 | The Perfect Guy | $57,017,904 | $3,246,901 | $60,264,805 |
| 1,372 | 2013 | Escape From Planet Earth | $57,012,977 | $17,143,633 | $74,156,610 |
| 1,373 | 2005 | Memoirs of a Geisha | $57,010,853 | $104,500,000 | $161,510,853 |
| 1,374 | 1963 | Cleopatra | $57,000,000 | $14,000,000 | $71,000,000 |
| 1,375 | 1992 | Beethoven | $56,979,478 | $0 | $56,979,478 |
| 1,376 | 1998 | The Man in the Iron Mask | $56,968,169 | $0 | $56,968,169 |
| 1,377 | 2017 | Dunkirk | $56,957,597 | $55,178,785 | $112,136,382 |
| 1,378 | 2000 | Shanghai Noon | $56,932,305 | $14,257,530 | $71,189,835 |
| 1,379 | 2011 | Midnight in Paris | $56,816,662 | $106,021,711 | $162,838,373 |
| 1,380 | 2012 | American Reunion | $56,758,835 | $180,040,376 | $236,799,211 |
| 1,381 | 2008 | Saw V | $56,746,769 | $61,463,009 | $118,209,778 |
| 1,382 | 1998 | A Civil Action | $56,709,981 | $0 | $56,709,981 |
| 1,383 | 1968 | 2001: A Space Odyssey | $56,700,000 | $12,000,000 | $68,700,000 |
| 1,384 | 1967 | Guess Who's Coming to Dinner | $56,700,000 | $0 | $56,700,000 |
| 1,385 | 2002 | The Time Machine | $56,684,819 | $42,298,771 | $98,983,590 |
| 1,386 | 1996 | Executive Decision | $56,679,192 | $65,400,000 | $122,079,192 |
| 1,387 | 2013 | 12 Years a Slave | $56,671,993 | $124,353,350 | $181,025,343 |
| 1,388 | 2001 | Rat Race | $56,607,223 | $30,000,000 | $86,607,223 |
| 1,389 | 1995 | Clueless | $56,598,476 | $0 | $56,598,476 |
| 1,390 | 1993 | Tombstone | $56,505,065 | $0 | $56,505,065 |
| 1,391 | 2015 | Joy | $56,451,232 | $44,682,827 | $101,134,059 |
| 1,392 | 2006 | The Queen | $56,441,711 | $72,444,162 | $128,885,873 |
| 1,393 | 2002 | Drumline | $56,398,162 | $0 | $56,398,162 |
| 1,394 | 2003 | Once Upon a Time in Mexico | $56,330,657 | $41,082,870 | $97,413,527 |
| 1,395 | 2016 | Me Before You | $56,245,075 | $149,200,000 | $205,445,075 |
| 1,396 | 1991 | Boyz n the Hood | $56,190,094 | $10,540 | $56,200,634 |
| 1,397 | 2003 | Just Married | $56,127,162 | $45,437,773 | $101,564,935 |
| 1,398 | 2015 | The Fantastic Four | $56,117,548 | $111,000,000 | $167,117,548 |
| 1,399 | 2005 | White Noise | $56,094,360 | $36,000,000 | $92,094,360 |
| 1,400 | 2001 | A Knight’s Tale | $56,083,966 | $44,538,620 | $100,622,586 |
The Domestic Market is defined as the North American movie territory (consisting of the United States, Canada, Puerto Rico and Guam).
This table is updated daily to reflect the latest studio reports.
Note: This chart is not adjusted for inflation.