The latest SD card image has had some issues with the fake-hwclock and many of the problems encountered with using a read-only file system on a Pi are down to incorrect timestamps as either a network connection isn't available or the files are locked to read-only and cannot be updated. The latest image has the fake-hwclock.data file relocated to the writable data partition and a symlink in it's place. in most cases this works well but can cause an issue if events don't happen exactly as expected at the right times during boot up.
Having helped a few users with this issue I did some research and this is what I found.
When the ntp daemon is initiated at boot-up it will attempt to correct the current system time that was previously loaded from the fake-hwclock.data file using ntp servers. Only the initial synchronization attempts are able to overcome a difference greater than 1000 secs, if any subsequent attempts discover a difference of 1000 secs or more the ntp daemon will terminate and no longer monitor or correct the time as it assumes user intervention is required to resolve a greater issue than a minor alignment. This means if there is no internet connection at the time of the initial attempts, it is then unable to resolve any time difference unless the ntp daemon is restarted and performs a successful initial sync.
Providing the time difference remains less than 1000 secs the npt daemon will then continue to check and resync the time approximately every 5-6 mins (usually cycling 4 servers, each checked at up to 1024secs intervals). this time is held in memory and not affected directly by read-only access.
A cron job is run hourly to save the current system time to fake-hwclock.data. This does require write access so in read-only mode the "saved" time remains unchanged (unless rpi-rw is used and then left in RW mode), in the last pre-configured SD card image the fake-hwclock.data file was moved to the RW "data" partition and can be written to at anytime.
If the Pi is shutdown or rebooted correctly the current system time is saved to fake-hwclock.data during the shutdown process as well, depending on if write access is available at the time.
This operation generally doesn't present a problem, but what is happening is the Pi is not ready to follow symlinks during the early stages of booting up so it cannot see the symlinked "data" partition's copy of fake-hwclock.data, which then results in a unix time of 0 (1970) as the file appears empty and if the internet connection is not then available for that first time resync, ntp will exit at a subsequent check as 1970 til now is over 1000 secs.
Only rebooting the Pi or restarting ntpd with an active internet connection can correct the time once this happens.
I therefore recommend reverting fake-hwclock.data to its former position (only if it's located on the data partition) and then editing 2 files to momentarily provide write access to enable that file to update at hourly intervals and again at shutdown.
to do this first we need to switch from read-only mode to write mode using
rpi-rw
1) Delete the symlink and move the relocated fake-hwclock.data back to it's original location
sudo rm /etc/fake-hwclock.data
sudo mv /home/pi/data/fake-hwclock.data /etc/fake-hwclock.data
This should ensure the file can be found at boot time so the Pi shouldn't revert to unixtime 0 (1970) instead it should use the last saved time and date temporarily until the time is corrected by ntp (this may not be accurate but it should be more recent than 1970).
2) Enable the hourly cron job to back-up the current time by temporarily remounting the partition as RW.
sudo nano /etc/cron.hourly/fake-hwclock
replace this line
fake-hwclock save
with these 3
sudo mount -o remount,rw /dev/mmcblk0p2 /
fake-hwclock save
sudo mount -o remount,ro /dev/mmcblk0p2 /
By enabling the time to be back-up every hour any power interruptions can only result in a 1hr maximum (plus downtime) error in time at reboot until the time is corrected by ntp. Again not accurate but better than running full RO and going back to some much earlier point in the past or running full RW and risking damage to the SD card or file system.
3) Also enable the fake-hwclock service to save the current time at shutdown the same way.
sudo nano /etc/init.d/fake-hwclock
and replace this line
fake-hwclock save;;
with these 3
sudo mount -o remount,rw /dev/mmcblk0p2 /
fake-hwclock save
sudo mount -o remount,ro /dev/mmcblk0p2 /
;;(edited to correct ";;" location)
Whilst nothing can be done if there is a sudden loss of power this edit will mean a graceful reboot should only lose seconds.
These three steps should restore the Pi's timekeeping to it's former Raspbian standard and overcome any issues imposed by mounting the file system as read-only.
Restart the ntp daemon and return to read-only mode
sudo /etc/init.d/ntp restart
rpi-ro
Additionally as the accuracy of the data we collect is dependent on an accurate timestamp I suspect a 4th step may be needed to improve the Pi's timekeeping, this additional step could counteract delayed network connections at start up caused by a poor internet connection or even just a router taking longer than the Pi to reboot and get a connection after a power outage.
4) Create an init script to run a small utility at start-up to keep testing for an internet connection every 60secs or so and when successful, restart the ntpd service to initiate the first run again and then exit. there will be no further use for it unless the Pi is rebooted or the time gets over 1000 secs out of sync (if that's happening theres a bigger issue somewhere).
5) Possibly create a cron entry to re-start the utility and therefore the ntp daemon periodically
I have started writing a script for step 4 and will post it when i have completed it and done some testing, hopefully we can resolve this fully before the next image is compiled.
If you are not totally confident with doing these changes please hang on until a couple of other users have tried it and hopefully confirmed it's ok, I see no reason for it not to work but I really would not like to be responsible for any corrupted or loss of data.
So any volunteers ?
Paul
Re: Read-only image time issues
Something doesn't sound quite right there. It's true that ntpd will only give you one shot at a jump greater than 1000 secs(*), but you don't spend that one-shot until it has successfully set the time. If ntpd has successfully changed the kernel time by more than 1000 seconds and it later decides it needs to do so again, then it exits. But if it can't currently reach any servers, it just waits until it can.
You can verify that you don't waste your one-shot during network outages with this simple experiment:
1. stop ntpd
2. manually set the time to 1970
3. pull the ethernet cable
4. start ntpd
5. wait as long as you like
6. plug the ethernet cable back in
ntpd will then jump the kernel time all the way forward from 1970 to the correct time. On some systems it might take a while, depending on local jitter. On one of my machines (an Asus netbook) it takes about 5 mins before it decides it's going to trust the NTP servers. But on everything else I've tried (including 2 Raspberry PIs running various distros) it latches on after just a couple of updates.
(*) you only get the one-shot if you're running ntpd with the -g option, which most Linux distros do. If you repeat the above experiment without the -g, ntpd will exit after step 6 when it hears from the NTP servers and decides the jump is too large. See the man page excerpt below. Most Linux distros set the ntpd switches in /etc/default/ntp
What do you do with the driftfile when you mount readonly? On most Linux distros it lives in /var/lib/ntp, but it's location is specified in /etc/ntp.conf with the driftfile directive. ntpd writes to that every hour (unless the value it wants to write hasn't changed).
man ntpd:
-g Normally, ntpd exits with a message to the system log if the offset exceeds the panic threshold, which is
1000 s by default. This option allows the time to be set to any value without restriction; however, this can
happen only once. If the threshold is exceeded after that, ntpd will exit with a message to the system log.
This option can be used with the -q and -x options.
man ntp.conf
driftfile driftfile
This command specifies the name of the file use to record the frequency offset of the local clock oscillator.
If the file exists, it is read at startup in order to set the initial frequency offset and then updated once
per hour with the current frequency offset computed by the daemon. If the file does not exist or this command
is not given, the initial frequency offset is assumed to be zero. In this case, it may take some hours for the
frequency to stabilize and the residual timing errors to subside.
The file format consists of a single line containing a single floating point number, which records the fre‐
quency offset measured in parts-per-million (PPM). The file is updated by first writing the current drift
value into a temporary file and then renaming this file to replace the old version. This implies that ntpd
must have write permission for the directory the drift file is located in, and that file system links, symbolic
or otherwise, should be avoided.
Re: Read-only image time issues
I was of the same opinion that "Something doesn't sound quite right there" despite my doubt, my understanding was the ntpd -g was not repeated after a successful internet connection was established, although I could not find anything official to absolutely verify this one way or the other I did trawl through many forum discussions, guides and tutorials to arrive at that conclusion, it makes far more sense to only adjust the time after successfully confirming the correct time with live ntp servers over the internet.
This however doesn't explain why several users have had a system time/date of 1970 for prolonged periods of time (days) despite having ssh access and posting data to emoncms.org, as barmy as my findings seemed it fitted the issue.
I could only find limited info on the Raspbian ntp implementation and the fake-hwclock's involvement. What I couldn't establish specifically was if the loading of the "fake time" at boot effected the one-shot.
I'm not able to do the tests at the moment but will try them at a later date, in the meantime I will assume you are correct as you appear to know the system well and it's certainly more feasible for it to work that way. If the one-shot is retained until a working internet connection is available that's great as my suggested steps 4 & 5 will not be required.
The main issue though, is that the Pi is apparently unable to read the symlinked copy of the fake-hwclock.file at boot up which results in a 0 timestamp, this seems to get rectified if there is an internet connection available, but data can be disrupted in the interim even when there is a good internet connection, obviously the longer it takes the worse the impact will be.
See attached log1 this is an excerpt from a emonhub.log from a Pi running the latest SD image and has a perfect internet connection. The timestamps are good to begin with and @2014-09-21 14:07:19,967 a reboot command is issued, emonhub closes, the Pi reboots and emonhub restarts, emonhub's log messages start at 1970-01-01 01:00:58,039 (unixtime 58.039) and around 12 seconds later the time sets to 2014-09-21 14:08:44,914. The fake-hwclock on this Pi was on the writable data partition and would of been saved at shutdown, this was confirmed to be the case that the file is being written but then ignored at boot.
Log2 is from a read only system that did not have the fake-hwclock.data file on a writable partition. however the Pi was in RW mode until it was restarted at around midnight the previous evening (power outage) therefore the "saved time" was 00:17:02. Following a reboot at around 15:50 the initial time is the "saved time" not 1970 for a few seconds and then gets corrected very quickly by ntp.
Log3 is the second Pi again a few minutes later, it is put in RW mode before rebooting again and it clearly saves the time at shutdown and loads it as expected at boot.
The suggested steps 1 to 3 will allow the fake-hwclock.data file to be found at boot, plus allow it to be saved hourly and at shutdown,
I don't think the drift time is catered for in the read only image, the file is located /var/lib/ntp/ntp.drift which is not currently writable, I'm guessing the impact of that is quite minimal, but I may look into it once we get the Pi to reliably report the correct millennium :-)
Paul
Re: Read-only image time issues
I'm not particularly familiar with fake-hwclock (none of my distros include it). If I get time tomorrow I'll play with it. From what I can find on the web, it's a way of loading the kernel time with an approx time (+/- an hour) rather than using 1970, when ntp hasn't done its stuff yet. Personally, I prefer my clocks to display 1970 when they're wrong, that way I know they need attention, but each to their own.
When one is stuck in the 70's have you been able to determine whether or not ntpd is still running? If it is running and just not syncing, ntpq is an good utility for poking around inside ntpd's state and looking at its associations and peers.
I don't think the drift time is catered for in the read only image, the file is located /var/lib/ntp/ntp.drift which is not currently writable, I'm guessing the impact of that is quite minimal,
I'm not sure if it's relevant or not, but I suspect if you did have a valid driftfile, ntpd might latch onto the NTP servers faster (in the case where the network is up).
P.S. Do you have ntpdate installed? That will immediately force the system time to what the NTP servers say, without waiting around for ntpd to sync up (again only useful in the case where the network is up).
Re: Read-only image time issues
Fake-hwclock seems simple enough http://manpages.ubuntu.com/manpages/raring/man8/fake-hwclock.8.html#contenttoc5 (not using ubuntu, but fake-hwclock seems quite uniform across distros) it just saves the current time to file hourly by cron and at shutdown. That time is then supposed to be loaded at boot. I assume the symlink is the hurdle.
I can see the benefit there and would normally agree but then I would have to implement some sort of test before allowing emonhub, emoncms or any other time recording software to start and that could result in lost data rather than slightly inaccurately timestamped data, (always seem to be deciding the lesser evils)
Because emonhub either buffers the incorrectly timestamped data until a network connection is available or in the case of a local emoncms which doesn't need a network connection to receive from a local emonHub, the data with the wrong timestamp can play havoc on some feeds. for example a whaccumulator that suddenly goes back in time by a few days or hours due to a reboot,
Having now read a bit about the drift file I think you are right the valid drift file would result in a quicker correction as an absent file causes lengthy calculations to begin that can cause huge delays to corrections. I have checked all my P's and they all appear to have a file but it is not writable on the RO ones, so i assume the file is created during the initial set up stages and only updated if it happens to try and update while I have the Pi in RW mode for another reason. So I guess in my cases the Pi's will quickly establish a slightly inaccurate time (+/- 20ppm is not alot) another lesser evil trade off!
Ideally I would like to make the drift file RW momentarily to save but cannot find where this is done. It would be pointless symlinking it to "data" as would creating a file in RAM at boot as this would initiate the calculating at each boot.
I don't have ntpdate installed as apparently it will not work after ntpd is running so I would effectively only mimic what ntpd -g does currently unless ntpd is stopped and restarted which in itself would rerun the "one-hit" and would need a trigger once the network is up which is where I was going with step 4.
Having only read up on this over the weekend I haven't established if ntpd is running when these issues occur, I don't actually experience the issues on my Pi's but several other users have and as the issue seem to always be perceived as an emonhub error I though I would have a go at resolving it , but to do that I need to set up a Pi running the pre-configured image with a screen and keyboard to be able to monitor the effects of no network as ssh will include a network connection unless I disconnect my whole network from the outside world.
Re: Read-only image time issues
Ideally I would like to make the drift file RW momentarily to save but cannot find where this is done.
It's rewritten by ntpd every hour (regardless of whether it previously existed). So in the case of a virgin system you've just installed, the file will first appear one hour after you first install and run ntpd.
Re: Read-only image time issues
I don't have ntpdate installed as apparently it will not work after ntpd is running so I would effectively only mimic what ntpd -g does currently unless ntpd is stopped and restarted
On all the machines I have ntpdate installed, it puts a script called ntpdate into /etc/network/if-up.d (scripts in that directory get executed when an interface comes up). That script stops ntpd, runs the ntpdate binary, and then restarts ntpd.
ntpdate is a lot more brute-force than ntpd. ntpdate pretty much grabs the time from an NTP server and slams it into the system clock, with very few questions asked. By comparison, ntpd sits in the background listening to NTP server messages, runs filters on them, and if you're lucky, will eventually decide to sync to one of them.. but this can take tens of minutes. From there on in, it's very good at tracking those servers and keeping your system time extremely accurate.
So the two work very nicely together. ntpdate will force your system time to be fairly correct pretty much immediately, and ntpd will then get it (and keep it) extremely correct after that.
Here's an example log of me bringing my desktop machine out of suspend at 21:02:
Oct 6 21:02:13 ntpd[21608]: ntpd exiting on signal 15
Oct 6 21:02:28 ntpdate[22939]: adjust time server <IP_ADDRESS> offset 0.373472 sec
Oct 6 21:02:28 ntpd[22966]: ntpd<EMAIL_ADDRESS>Tue Jun 5 20:12:08 UTC 2012 (1)
Oct 6 21:02:28 ntpd[22967]: proto: precision = 0.111 usec
You can see it killed ntpd, then it ran ntpdate to brute-force my system time (which had drifted by 0.37 secs while suspended), then it restarted ntpd. ntpd then took a full 4 minutes before it sync'd to an NTP server (not shown in the logs). You can see ntpdate took just 15 seconds to fetch the time and fix my system time.
Re: Read-only image time issues
Ok so ntpdate could force a correction at anytime as and when the network comes up,
How would that be affected by an active network connection without internet access ? for example a power outage following which the Pi is quick to reboot and a router could provide an ip address but an external connection to the internet takes a bit longer to establish. The script I had written tests connectivity by trying to open a webpage, I'm guessing the if-up.d could get caught out here.
Although once triggered my script restarts ntpd I suppose it could be used to run ntpdate instead.
The ntp.drift file is still equally important regardless of how the initial time is established I imagine, if the file is missing the calcs will take considerably longer to fine tune the time.
I can only think of one way around it and that is to create a /var/lib/ntp directory in RAM using an entry in the fstab ( currently done for /var/logs) and as the Pi is booting but before the ntpd is started a ntp.drift.BAK file is copied to /var/lib/ntp/ntp.drift for ntpd to find and keep updated and the hourly cron job that currently saves fake.hwclock.data gets extended to save the ntp.drift file to ntp.drift.BAK and the same for saving at shutdown so that there is always a recent drift value available at boot up.
Even if a RTC was fitted to removed the need for fake-hwclock, the drift file would probally still need to be functional.
If there is a recent fake-hwclock.data entry and a recent ntp,drift file is there a need for the ntpdate as well do you think? ntpd's gradual changes have their advantages as well eg less chance of skipping a scheduled cron job or jumping forwards or backwards in time while mid measurement.
Re: Read-only image time issues
There again if ntpdate is used the drift at that time would be 0 so a drift file could just be created with a zero value to prevent the longer ntp calcs starting, then there would be no need to back it up just the RAM copy would do.
Re: Read-only image time issues
Ok so ntpdate could force a correction at anytime as and when the network comes up, How would that be affected by an active network connection without internet access ?
Just as your script is polling a webpage, ntpdate is polling an NTP server. In the man page you'll find several switches that let you control its behaviour in that regard:
-p samples
Specify the number of samples to be acquired from each server as the integer samples, with values from 1 to 8
inclusive. The default is 4.
-t timeout
Specify the maximum time waiting for a server response as the value timeout, in seconds and fraction. The value
is is rounded to a multiple of 0.2 seconds. The default is 1 second, a value suitable for polling across a LAN.
If there is a recent fake-hwclock.data entry and a recent ntp,drift file is there a need for the ntpdate as well do you think? ntpd's gradual changes...
Utlimately I guess that's a design decision for your server, but in general I think the answer is yes. The options' features are:
fake-hwclock: instant time setting, no network required, accuracy to within about an hour
real-hwclock: instant time setting, no network required, accuracy to within tens of seconds
ntpdate: ~15 secs to set the time, network required, accuracy to within hundred of msecs
ntpd: tens of minutes to set time, network required, accuracy to within one or two msecs
On an always-on server the first 3 only happen after a major event, i.e. a reboot or a LAN failure. The general consensus of most distro designs is you want to get the server time as accurate as you can as quickly as you can, and then fine tune it, so they do real-hwclock, ntpdate, ntpd in that order. Given you have to use fake-hwclock with its incredibly poor accuracy, I would think ntpdate would be even more important.
But you're right to consider all the possible failure modes and how you want your server to behave under those conditions. For example, if the ethernet cable has fallen out of the RPi the interface will never come up, but if it's receiving data via RF and has local storage, you might still want it to log. Or your DSL service might be down, so the interface comes up, but ntpdate and ntpd will never get you the correct time.... etc. etc.
There again if ntpdate is used the drift at that time would be 0
The driftfile doesn't contain the difference between your time and the NTP server's time (that difference is known as "offset" in NTP parlance). The driftfile contains the frequency adjustment that needs to be applied to your local crystal in PPM. As ntpd runs it effectively calibrates your crystal for you. That has two benefits:
. if you fall off the network, time will continue to advance based on your adjusted crystal
. next time you restart, ntpd can immediately be using your adjusted crystal while it tries to sync to the servers
a drift file could just be created with a zero value to prevent the longer ntp calcs starting
A driftfile containing 0 is identical to a missing driftfile to ntpd (see man page entry from an earlier post above).
Re: Read-only image time issues
I wonder if there's any path through that maze of scripts that can end up writing 1970 to the fake h/w clock, and then periodically loading the fake h/w clock into the system time, behind ntpd's back. That would be a way to use up your one-shot big jump.
Re: Read-only image time issues
As I recall, the cases that had 1970 stick for a prolonged time were run for a while without an internet connection. Although unsure of the exact details and durations etc I suspect the symlinked fake-hwclock was not accessible at boot so the 0 unix time stamp was loaded and then if no working internet connection was provided for the ntpd to rectify the time within 1 hour of booting up the 1970's time and date would get saved to fake-hwclock and there have been instances where the fake-hwclock.data has been confirmed as a 1970's timestamp.
The steps 1 to 3 above are definitely going to over come the initial 1970's problem and in the majority of cases, the Pi will function as expected.
A sturdier solution is still required for Pi's with weak or intermittent network connections, like ntpdate, my script or both maybe, I'm not sure until I can get a test rig set up to try a few things. The ntpdate script may poll the ntp servers once it has been triggered by the initial network connection but that may not be when the internet connection is established, by polling a site until a reply is obtained and then running ntpdate the time is sure to be corrected unless the connection is lost again within a second or 2 and not regained.
Although I was aware the drift value wasn't the difference. I had thought it was the required frequency correction to align the current and correct times gradually and therefore would be zero if a correction is forced. So the drift value ideally needs to be made "editable" using something like previously suggested, but as a last resort could be left as is so that it is always micro seconds out but doesn't restart the lengthy initial calcs which could delay any adjustments being made at all.
I think as the pre-configured image will have a date already set in fake-hwclock it would be good advice for users to initially connect to the internet for a period or to set the clock manually and reboot or run sudo fake-hwclock save to refresh the fake-hwclock ensuring no matter what the Pi will not then revert back before that date.
Re: Read-only image time issues
Paul,
I can confirm your 1st para above; this is precisely what I've validated to be happening for me here.
And, it's a good idea that to 'deploy' this emonbase, we need an install doc or procedure that requires the user to get the date correct as they set it up.
I expect to be going to my 'offline' emonbase tomorrow so it'll be interesting to see if I can get it working without access to internet. My worry would be if there were a power outage, it'll need manual intervention to recover.... Admittedly this completely offline use case is a severe example but we all like a challenge :-)
Thanks a lot for going to such trouble investigating this very interesting issue.
I've implemented your steps 1-3 on another system I have here. The 'last updated' status on the input page is performing a lot better ; i.e. there are 2 nodes, and both are updating very quickly. I was seeing long delays for one of the nodes. There are of course other variables, such as whether doors are closed etc but it's definitely much 'cleaner' just from watching the input page update.
Eamonn.
Re: Read-only image time issues
Thanks for the feedback Eamonn, In the case of your offline Pi, doing steps 1 to 3 will be enough to allow the Pi to count from the time and date you set and store that value at hourly intervals and at shutdown. This means the accuracy will not be corrected and may loss or gain a few seconds, not the end of the world. But a sudden power loss will result in a time loss of up to 1 hr plus outage time, so a prolonged outage will disrupt the validity of the ongoing data as would a series of momentary interruptions eg a momentary outage just before the "hourly save" will set the timekeeping back an hour to the previously saved time.
Unless you can be sure the power is totally reliable I would recommend a RTC as a power outage may be un traceable and invalidate all your data as the Pi is none the wiser and keeps logging from the last saved time after a reboot as if time stood still, days weeks or months later you would just see a time or date difference but the data probably won't show the outages and there is no way for it to alert you at the time either.
Re: Read-only image time issues
agreed ; I do need to get a config with the RTC for the offline scenario.
Eamonn
Re: Read-only image time issues
I had thought it was the required frequency correction to align the current and correct times gradually and therefore would be zero if a correction is forced.
Ah ok. ntpd is constantly trying to fix the frequency and fix the offset. The driftfile is where it keeps track of the frequency fix it's calculated for this machine. So even if you somehow did get the system time perfect such that no offset adjustment was currently required, the value stored in driftfile wouldn't change.
Whether or not it can write the file, internally it continues to use the value it wants to use (and would have written). So not being able to write to the file will only affect things next time it starts up. Instead of starting with the frequency tweak it's learned over the months/years it's been watching, it has to start over. On all the machines I've ever looked at, once it's been up for a few days (months?) the contents of that file barely changes. But it does vary a lot from machine to machine of course.
So maybe you can design something with your readonly Vs read-write filesystem to take advantage of that, i.e. you could perhaps get away with writing it once after ntpd has been sync'd for a few days, and then burning that value into the readonly version for use from them on (but on a per-machine basis, not globally)
Re: Read-only image time issues
Finding a point to intervene in the drift file saving that isn't too complex and minimal opportunity to introduce side effects could be tricky. Where as the fake-hwclock uses cron, so a couple of simple edits to remount as RW momentarily works, it appears the ntpd code does the hourly drift file save directly, that is an assumption based on the fact I couldn't find a cron for it and I'm unable to view the ntp code to confirm.
I'm not sure if I would be able to query the drift value from ntp and then write it to a file, where as if the ntp.drift file is moved to RAM ntpd can update it as usual and that file can then just be copied to a RO location that can be copied back at boot time.
Since the ntpd saves once an hour only if the value has changed I feel that is probably the safest route for the "fix" to take and amalgamate the hourly back-up of the current time and drift value (if != existing back-up) into one momentary remount cycle.
I need to experiment a bit really...
Re: Read-only image time issues
it appears the ntpd code does the hourly drift file save directly
Yes, it does.
I'm unable to view the ntp code to confirm
http://www.ntp.org/downloads.html
I'm not sure if I would be able to query the drift value from ntp
You can do it from C by calling ntp_adjtime(), or from a script with the kerninfo command in ntpdc and parsing the output for the item labeled "pll frequency":
# ntpdc -c kerninfo | grep "pll frequency"
pll frequency: -11.870 ppm
#
# more /var/lib/ntp/ntp.drift
-11.869
#
In the above example, my ntp.drift file is about 40 minutes old and you can see it's already changed its mind (ever so slightly) about what the value should be.
[EDIT]
I feel that is probably the safest route for the "fix" to take
Don't interpret the details I've added above as an attempt to talk you out of your proposed solution. If anything, I probably agree with you that it's the safest route. I was just filling in the bits you were unsure about so you've got plenty of options to choose from.
Re: Read-only image time issues
Fantastic so something as simple as
would just overwrite the existing value in ntp.drift, if run as root by cron.
and If that could be improved further to only over write if the value is not equal to the existing contents I think that would be much better than creating a file in RAM and copying at boot.
Re: Read-only image time issues
To echo Eamonn, thanks a lot for looking into this Paul, dbC, I think im just about following.
Re: Read-only image time issues
So something like this one line added to both the fake-hwclock hourly cron and the ntpd shutdown script
should save the drift value to file, but only if it has changed, while the fake-hwclock routine has the file system mounted as RW momentarily.
shell isn't my strong point so I welcome any corrections if that syntax can be improved, although I have just tested the logic and it seems to function correctly I have not actually tried it in place yet.
Re: Read-only image time issues
I still don't have a Pi set up that I can remove the network connection from (as I run them headless), but I have now added these mods to a read-only Pi to see if it works ok normally.
Step 2 adds a neat safety function as every hour the system will be switched to RO regardless of whether it is RO or RW at the time, handy for those of us that never remember to use rpi-ro after making changes :-)
Re: Read-only image time issues
A couple of corner cases you might want to test against:
1. if ntpd has exited for some reason, then ntpdc will output an error: "ntpdc: read: Connection refused", so you'd want to make sure you don't clobber your good drift file with a bad one (or null one) in that case
2. it's also quite legal for the drfit file not to exist, in which case you want to create it, so you'd want to make sure your "has it changed?" test doesn't prevent that.
While it's true that ntpd does include that "has it changed?" optimisation before it rewrites ntp.drift every hour, in practice I think it's a code path rarely exercised. On all my machines, I can't find a single one where ntp.drift is older than 1 hour. Even on machines that have been up a long time, by the time an hour has come around it almost always wants to tweak the value, even if only by 0.001 as in the example I pasted above. So if it makes it easier (or more robust) you could drop that test and I suspect it would make little difference to how often the file is written.
Re: Read-only image time issues
You're right, if the drift value is most likely going to be different more often than not, then the "if changed" may introduce more issues than it helps ie not creating the file if it doesn't exist yet. i like the simplicity of "just copy it regardless" although as you point out ntpd may not be running for example if a difference of 1000 secs is detected after the "one big step" it will close.
Would a simple check to see if the ntpd.pid file exists be sufficient ?
I am also considering creating a cron and init script called something like "ntp-backup" so that the existing file(s) can be deleted, disabled or just ignored so that our mods are independent and unlikely to be altered by updated packages or in the case of someone fitting a RTC and removing/deleting the fake-hwclock as per most guides instruct, the drift file will still work and if fake-hwclock is re-installed/re-enabled it will work with the RO file system again.
Re: Read-only image time issues
Thanks for that
now i am seeing a timestamp on my inputs, however im not getting any reports from my CT inputs...
here is an output from my logs, i have increased the log level :
pi@raspberrypi:~$ tail /var/log/emonhub/emonhub.log
2014-10-09 13:11:35,983 DEBUG 8 Append to 'emonCMS' buffer => time:<PHONE_NUMBER>.92, data: [10, 0, 0, 0, 0, 25433, 0], ref: 8
2014-10-09 13:11:36,089 INFO Sending: http://localhost/emoncms/input/bulk.json?apikey=E-M-O-N-C-M-S-A-P-I-K-E-Y&data=[[1412860295.92,10,0,0,0,0,25433,0]]&sentat=1412860296
2014-10-09 13:11:37,251 DEBUG Receipt acknowledged with 'ok' from http://localhost/emoncms
2014-10-09 13:11:46,763 DEBUG 9 NEW FRAME :<PHONE_NUMBER>.76 10 0 0 0 0 0 0 0 0 103 99 0 0
2014-10-09 13:11:46,768 DEBUG 9 Timestamp :<PHONE_NUMBER>.76
2014-10-09 13:11:46,771 DEBUG 9 Node : 10
2014-10-09 13:11:46,773 DEBUG 9 Values : [0, 0, 0, 0, 25447, 0]
2014-10-09 13:11:46,803 DEBUG 9 Append to 'emonCMS' buffer => time:<PHONE_NUMBER>.76, data: [10, 0, 0, 0, 0, 25447, 0], ref: 9
2014-10-09 13:11:46,906 INFO Sending: http://localhost/emoncms/input/bulk.json?apikey=E-M-O-N-C-M-S-A-P-I-K-E-Y&data=[[1412860306.76,10,0,0,0,0,25447,0]]&sentat=1412860306
2014-10-09 13:11:47,385 DEBUG Receipt acknowledged with 'ok' from http://localhost/emoncms
the mains power voltage seems to work fine, just the CT come back with no readings.
please advise
thanks
Re: Read-only image time issues
Would a simple check to see if the ntpd.pid file exists be sufficient ?
No, the pid file only gets cleaned up if ntpd is shutdtown gracefully by management. If the daemon just quietly exits, the stale pid file will remain around forever (well until it's replaced by a new one next time it gets restarted). It's probably better to do a sanity check on dval before you write it to the file. In the case where the daemon is not running, dval should end up empty after your assignment because the grep will fail to find anything.
Re: Read-only image time issues
Ok so a more reliable check would be to just check $dval for content, makes sense and it's simpler, win-win ! I'll take another look at doing this over the weekend hopefully and get a spare screen down from the loft to run some tests, thanks, I really appreciate all your help with this!
Re: Read-only image time issues
Although still being tested I have packaged this up so that it can be implemented with one command from the home directory when in RW mode (rpi-rw) and so far, it seems to work ok.
Re: Read-only image time issues
nice one. spoiling us . .... Thanks!
Re: Read-only image time issues
No worries!
Let us know how you get on if you use it, I've tidied up some permissions etc this morning and tested some more, but you can't beat real world testing.
Re: Read-only image time issues
I've modified this a little to be less sensitive to where it is initiated from. it does still need to be in "write" mode before running
Re: Read-only image time issues
doh!


Introduction
In this work, we used the
Candoia to answer our research questions. Candoia enables
developing, customizing and adopting mining software repository(MSR) tools as application.
In this document we provide all Java source code, Candoia application code used to generate the data for this study.
We also provide the output of running these java programs, which is a chart used for this study. These results were further processed to present
them in the paper in a form easier to digest.
We provide these programs and data as an archival supplement to the paper, to allow researchers to understand our methodology
and, if they wish, use the data to investigate their own research questions.


Salmon Cookies
Sales per Store
Store Information
Name:
Minimum Customers per Hour:
Maximum Customers per Hour:
Average Cookies per Sale:
Submit
Store Sales per Hour Output:


The Tidy Messages table is a critical tool that you will use and should understand. It is what tells you what changes Tidy has made to your source document, and it communicates errors to you that it was unable to fix.
Follow these steps to become acquainted with the Tidy Messages table.
Observe the messages table, which consists of one or more rows of information indicating data about your document.
Each row will contain a Severity, a Location, and a Message Description, as described in the reference.
Clicking on row that contains a location will move the Source HTML view to that location and highlight it for you, as this example shows:


You've just created a simple Play Framework application with Slick!
This template will help you building a variety of Web applications.
Skalholt is a code generator. You will be able to easily generate source codes for Play Framework.
Once the application has been compiled and the server started, your application can be accessed at: http://localhost:9000
You can execute the Skalholt code generator from "skalholt-ui.bat".It generate source code from DB schema.
The flow of using Skalholt is as follows.
Please refer to Project Page.
http://tech-sketch.github.io/skalholt/


Chester Billiards League
2019-2020
Cayzer Cup Tables 2020
Group Tables
| GROUP A |
| Pos |
Team |
Played |
Points |
Aggregate |
| 1 | Upton A | 6 | 10 | 430 |
| 2 | Ewloe A | 6 | 8 | -58 |
| 3 | Upton C | 6 | 6 | -205 |
| 4 | Ewloe B | 6 | 0 | -167 |
Upton A & Ewloe A Qualify for Semi Finals
| GROUP B |
| Pos |
Team |
Played |
Points |
Aggregate |
| 1 | Waverton A | 6 | 12 | 797 |
| 2 | Waverton B | 6 | 6 | -252 |
| 3 | Eastham HG | 6 | 4 | -34 |
| 4 | Upton B | 6 | 2 | -511 |
Waverton A and Waverton B Qualify for Semi Finals
Dress Code
Now that we are at the Semi-Final stage of the competition, please note that the dress code
comes into operation.
No Jeans, trainers or collarless shirts please.
In the event of a failure to comply with this rule, the offending player will forfeit his
game and his opponent will score 200 points.
The offending player will score the amount of his handicap.
Semi Finals
In view of the ongoing Coronavirus outbreak it has been decided to suspend
all games in the Billiards League until further notice.
Further details will follow as the situation develops.
| Date |
Time |
Home |
Score |
|
Score |
Away |
| TBA |
19:30 |
Upton A |
|
v |
|
Waverton B |
| TBA |
19:30 |
Waverton A |
|
v |
|
Ewloe A |


--- layout: documentation title: TTPsType ---
| Field Name | Type | Description |
|---|---|---|
| TTP0..n | TTPBaseType |
Characterizes a single cyber threat adversary Tactic, Technique or Procedure. This field is implemented through the xsi:type extension mechanism. The default and strongly recommended type is TTPType in the http://stix.mitre.org/TTP-1 namespace. This type is defined in the ttp.xsd file or at the URL http://stix.mitre.org/XMLSchema/ttp/1.1.1/ttp.xsd. |
| Kill_Chains0..1 | KillChainsType |
The Kill_Chains field characterizes specific Kill Chain definitions for reference within specific TTP entries, Indicators and elsewhere. |


{{define "plan"}}
Name
{{.Name}}
$/Month
{{.Price}}
{{.Name}} Plan
{{.}}
Are you sure ?
Changing your current plan will change the invoice immediately.


Hi <%- userName %> Where are you from?
Country
Please type your country.
City
Please type your city.
Next


{%- include footer.html %}
{% include scripts/jquery.html %}
{% include scripts/bootstrap.html %}
{% include scripts/masonry.html %}
{% include scripts/misc.html %}
{% include scripts/mathjax.html %}
{% include scripts/analytics.html %}
{% include scripts/progressBar.html %}


Life Reviewed by Mac VerStandig<EMAIL_ADDRESS>http://www.moviereviews.org 3 Stars (Out of 4)
It has been said more than once that imitation is the highest form of flattery. Hollywood works much the same way, as only the truly great films seem to find themselves being imitated directly or indirectly in future releases. The one question that remains is whether or not an imitation can be as good as or better than an original. Sequels certainly don't tend to be, and remakes are very mixed. But what about in a movie like Life, where deja vu exists with no less than 5 other films but some level of originality is still in place? Director Ted Demme seems to have proven so.
Ray (Eddie Murphy) is a low life pick-pocket who makes a "living" going from petty crime to petty crime. Despite being witty, he seems to hold very little close to him emotionally, except for his father's grave, and has little sense of the true meaning of life.
Claude (Martin Lawrence) is a far more proper man. He has a job lined up for himself at the bank, and is almost over some bad debts. Unfortunately, he too has no true ambition in life, and plans on spending his money on Yankees tickets, rather than settling down and getting married to a woman that clearly loves him.
Through a very odd set of events that seems to wander from crime to crime, in a slow and boring fashion, the two men end up being sentenced to life in jail together, for a crime they did not commit. It is here that the rest of the film is to unfold, and where Claude and Ray are to learn exactly what a life sentence really is, and what it can take away from you.
A mere five years ago, The Shawshank Redemption was released and it had enormous success. Obviously, there are some stark similarities between the two films. Both feature an innocent protagonist (In Life's case, protagonists) who are in for the ultimate punishment. And both films go on to give commentaries on the institutionalization that prison can cause, by showing an older inmate dealing with a reality that the place he is soon to depart, has become a home to him. But that is where the similarities to that production end.
Other films, like Mr. Saturday Night, show two men growing old together and dealing with the cards they have been dealt. But none do so as spectacularly as this one. There are also scenes lifted from, but not limited to, Stir Crazy, Murder in the First, and Spartacus. But none of those have any true similarities beyond the obvious settings (prison) and situations. So, despite being very similar, Life does manage to take it's own direction.
Another potential grim point for this film is the obvious lack of reputable actors in the primary casting. To date, Martin Lawrence has done nothing that would qualify him to play what ultimately is a serious role in a semi-dramatic film. Especially not one that transcends so many generations, and requires him to be as young as 20 and as old as 90. The same can be said of Eddie Murphy, who, despite some great comedic works over time, has done little of a decent dramatic nature. Fortunately, for the benefit of this film, they both turn out beautiful performances that are certainly their finest so far.
The finest acting out of the two leads comes in the later part of the film. As the two progressively age, and the wrinkles on their face multiply, you see how well Martin and Murphy have adapted to playing a far elder role in a very realistic manner. Not only do they change their mannerisms and voices, but you can truly sense a change in overall persona within their characters. That is certainly something you can't remember seeing before out of these two foul mouthed comedians.
One of the weaker elements of this film is the opening. It takes a while to get things into place, and the movie does a relatively poor job of entertaining the audience for the first 20 minutes or so, beyond some occasional humor. Also, in a classic, yet extremely cheap move, the movie skips some much needed details in order to finally get things going. Although you are pleased that after those 20 minutes a plot emerges that is consistent with the film, you still are left with a feeling that something is missing. One scene features the two men in a holding cell, optimistic that since they are truly innocent, they will be acquitted. The next scene is merely a judge slamming down a gavel, pronouncing the word "life." Now, I fully understand that the film has neither the time nor ability to explore a trial sequence, but this all could have been avoided by merely cutting the prior scene in the holding cell.
The ultimate question that can be posed is whether or not a film is quality entertainment. The answer in regards to this production is "yes." There is an element of fine acting by two actors with potential that the world did not previously know of. There is an unfortunate down side in the film's opening, but that is forgivable. Ultimately, the driving force behind Life is that it finally shows that imitation can be quality as well.
The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


Who are we seeking to become?
behavior will become part of you
We get what we invest in. The time we spend comes back, with interest.
If you practice five minutes of new, difficult banjo music every day
, you’ll become a better
banjo player. If you spend a little bit more time each day whining
or feeling ashamed
, that behavior will become part of you. The words you type, the people you hang with, the media you consume
The difference between who you are now and who you were five years ago is largely due to how you’ve spent your time along the way.
The habits we groove become who we are, one minute at a time.
A small thing, repeated, is not a small thing.
No Reboot Kernel patching
Why we need patching linux kernel?
- Kernel patches are released for a number of reasons, but fixing security holes is the most frequent reason. This is why it’s important to install the patch as soon as possible.
- Unlike other operating systems, Linux is able to update many different parts of the system without a reboot, but the kernel is different. Every running process integrates with the kernel intimately, so switching out parts of the kernel while it is running is quite risky. So, When the kernel is updated via a patch, the system needs to
reboot
“No Reboot” Kernel patching
- On the other hand, rebooting the computer is irksome, and in some cases, where uptime is important, it can be a real issue. This is why “no reboot” kernel patching has been a priority for many administrators.
- Some servers and critical real-time applications must not be taken down without advanced scheduling, even for a few minutes. This can be a pain when administrators need to keep the system secure and a patch is released to repair a newly discovered security hole. In this case, no-reboot patching becomes a real boon
- Recognizing this need, Red Hat has been working on
kpatch
, andSUSE
has been working onkGraft
. Both of these programs are designed to accomplish the same task, but they take a different approach and have different strengths. - But this doesn’t mean that system reboots are gone forever. Even on a system with the Linux 4.0 kernel, there will be security updates that still require a reboot, because there are other non-kernel components that can require patching, and some of these require a reboot as part of the process.
How DNS works
Without DNS, we would only be able to visit any website directly via IP address, no human-readable anymore
DNS: Basic concept and information
- DNS stands for Domain Name System
- Its purpose is to resolve and translate human-readable website name to IPv4 or IPv6 address
- It’s basically a large Database which resides on various servers around the world, that contains the names and IP address of various host/domains.
- Without DNS, we would only be able to visit any website directly via IP address, no human-readable anymore
- DNS works on both the TCP and UDP Protocols, port 53
How does DNS work?
Whenever you visit a domain such as github.com, the browser’s journey begins »>
Step 1: Request information
- TODO: Type
github.com
to address bar of Chrome, it’s a new request for browser
Step 2: DNS cache on Chrome
- TODO: DNS query on chrome’s cache, see
chrome://net-internals/#dns
- True: Return IP address of this domain for making a request
- False: Didn’t know it before –> call OS (step3)
Habit and Time management
Habit
Habit: something that you do often and regularly, sometimes without knowing that you are doing it. It takes 21 days to form a new habit:
- Do it For Yourself
- Know the Benefits
- Know the Pain
- Start Simple
- Make it Daily
- Stay Consistent
Time management
Most important thing: Work more when you’re in the zone. Relax when you’re not.
- It’s normal to have days where you just can’t work, and days where you’ll work 12 hours straight.
- Stop multi-tasking. It merely kills your focus.
- We’re always more focused and productive with limited time.
What is FLUSH TABLES WITH READ LOCK in MySQL?
Command:
mysql> FLUSH TABLES WITH READ LOCK;
What exactly does command do?
- Set the global read lock - after this step,
insert/update/delete/replace/alter statements
cannot run - Close open tables - this step will block until all statements started previously have stopped
- Set a flag to block commits


This bare-bones project template just creates a minimal
nbproject/project.xml
. No Ant build script is created.
In fact the "project" does not really do anything; it is essentially
a placeholder for some files that you can open and close like a project
for convenience, rather than using the Favorites tab.
You might use this template to create a non-code project to hold design documents or similar files. If you actually want an Ant-based freeform project, but want to use your own custom project skeleton (e.g. based on company-mandated build scripts or source structures), take a look at:
About Project Templates (requires use of NetBeans plug-in module development support, but is very flexible)


 Nisi, tempora!
This is the lightbox modal
Lorem ipsum dolor sit, amet consectetur adipisicing elit. Nisi, tempora!
This is the lightbox modal
Lorem ipsum dolor sit, amet consectetur adipisicing elit. Nisi, tempora!
This is the lightbox modal
Lorem ipsum dolor sit, amet consectetur adipisicing elit. Nisi, tempora!


Syn is used to simulate user actions. It creates synthetic events and performs their default behaviors.
The following clicks an input element with id='description'
and then types 'Hello World'
.
Syn.click({},'description')
.type("Hello World")
Syn is typically used to simulate user actions as opposed to triggering events. Typing characters
is an example of a user action. The keypress that represents an 'a'
character being typed is an example of an event.
While triggering events is supported, it's much more useful to simulate actual user behavior. The following actions are supported by Syn:
click
- a mousedown, focus, mouseup, and click.dblclick
- two click!
events followed by a dblclick
.[Syn.prototype.key key]
- types a single character (keydown, keypress, keyup).[Syn.prototype.type type]
- types multiple characters into an element.[Syn.prototype.move move]
- moves the mouse from one position to another (triggering mouseover / mouseouts).[Syn.prototype.drag drag]
- a mousedown, followed by mousemoves, and a mouseup.focus()
doesn't run immediately in IE.
If you provide a callback function to Syn, it will
be called after the action is completed.
Syn.click({},'description')
.type("Hello World", function(){
ok("Hello World" == document.getElementById('description').value)
})
You might have noticed the then method. It provides chaining so you can do a sequence of events with a single (final) callback.
If an element isn't provided to then, it uses the previous Syn's element.
The following does a lot of stuff before checking the result: Syn.type('ice water','title')
.type('ice and water','description')
.click({},'create')
.drag({to: 'favorites'},'newRecipe',
function(){
ok($('#newRecipe').parents('#favorites').length);
})

jQuery Helper

If jQuery is present, Syn adds a triggerSyn helper you can use like:
$("#description").triggerSyn("type","Hello World");
Every browser has very different rules for dispatching key events. As there is no way to feature detect how a browser handles key events, synthetic uses a description of how the browser behaves generated by a recording application.
If you want to support a browser not currently supported, you can
record that browser's key event description and add it to
Syn.key.browsers
by it's navigator agent.
Syn.key.browsers["Envjs\ Resig/20070309 PilotFish/<IP_ADDRESS>\1.6"] = {
'prevent':
{"keyup":[],"keydown":["char","keypress"],"keypress":["char"]},
'character':
{ ... }
}
Have we missed something? We happily accept patches. The following are important objects and properties of Syn:
Syn.create
- contains methods to setup, convert options, and create an event of a specific type.Syn.defaults
- default behavior by event type (except for keys).Syn.key.defaults
- default behavior by key.Syn.keycodes
- supported keys you can type.Syn is really the foundation of JavaScriptMVC's functional testing framework - FuncUnit. But, we've purposely made Syn work without any dependencies in the hopes that other frameworks or testing solutions can use it as well.
Creates a synthetic event on the element.
new Syn(type, options, element, callback) -> undefined
{Object}
{Object}
{Object}
{Object}
{}
Syn


|
Elektra
0.8.14
|
Every backend, except filesys
, was unable to represent full file system semantics. Capabilities described the differences between filesys
and another backend. (For POSIX file systems a similar technique is pathconf()
. It allows the user to query the capabilities of a specific mounted file system given by path.) Capabilities made it possible to implement a backend different from the way filesys
works and let the backend still have predictable behaviour. The user could query a backend if it was capable of a specific detail of file system semantics.
Capabilities were initially introduced to make backend development easier, because they also expressed the disabilities of a backend. For example, it was possible for a backend to claim that it is not aware of comments.
Getting a single key works well for filesys
. However, for configuration files the whole content must be parsed even for a single key. Capabilities were able to describe that the backend will always retrieve and store all keys and will not be able to retrieve and store individual keys. This restriction simplified the implementation of such backends considerably. Capabilities allowed us to implement fstab, passwd and hosts backends.
But we soon found the limits of capabilities. Capabilities were unable to describe:
No capabilities are used, but instead other plugins implement the limitations other plugins have.
The main problem was that the complexity was just moved to the applications. In the end, application developers had to understand what the backend can write out. They no longer had confidence that the storage was able to understand the full semantics. Elektra's core was unable to hide that fact. Capabilities did not turn out to make backend development much easier. Instead, Elektra was more difficult to use with capabilities. Capabilities are no longer part of Elektra for these reasons.


AQUATIK is a website template perfect for video streamers, models, aestheticians or any small business ventures.
It is a fun and futuristic template inspired by the ocean, and handcrafted to run fast and smooth on both phones and laptops.
You can add as many paragraphs as you'd like.
You can put a menu or whatever you like here. The possibilities are endless.
- Service 1 - $10
Soufflé soufflé chocolate chocolate cake sugar plum halvah carrot cake. Caramels danish carrot cake pudding biscuit sweet caramels chocolate bar fruitcake. Cotton candy cotton candy cheesecake cotton candy ice cream icing liquorice.
- Service 2 - $20
Tiramisu cookie brownie bear claw jelly caramels fruitcake jelly. Muffin icing pastry candy canes sweet pie danish. Macaroon lollipop candy jelly-o apple pie dessert candy sweet roll. Sesame snaps brownie croissant.
- Service 3 - $30
Candy croissant pudding cookie. Icing jelly-o icing ice cream candy icing cotton candy. Croissant cotton candy candy chocolate bar marzipan dessert halvah oat cake cookie.


Provides a high level API to interface over sources storing distance, dissimilarity, similarity matrices with matrix style extraction, replacement and other utilities. Currently, in-memory dist object backend is supported.
| Version: | 0.2.0 |
| Depends: | R (≥ 3.4.0) |
| Imports: | proxy (≥ 0.4.19), dplyr (≥ 0.7.4), assertthat (≥ 0.2.0), fastmatch (≥ 1.1.0), tidyr (≥ 0.8.0), factoextra (≥ 1.0.5), ggplot2 (≥ 2.2.1), broom (≥ 0.4.4), fastcluster (≥ 1.1.25), pbapply (≥ 1.3.4) |
| Suggests: | knitr (≥ 1.15.1), rmarkdown (≥ 1.4) |
| Published: | 2018-08-02 |
| Author: | KS Srikanth [aut, cre] |
| Maintainer: | KS Srikanth <sri.teach at gmail.com> |
| BugReports: | https://github.com/talegari/disto/issues |
| License: | GPL-3 |
| URL: | https://github.com/talegari/disto |
| NeedsCompilation: | no |
| Materials: | README NEWS |
| CRAN checks: | disto results |
| Reference manual: | disto.pdf |
| Vignettes: |
Introduction_to_disto |
| Package source: | disto_0.2.0.tar.gz |
| Windows binaries: | r-devel: disto_0.2.0.zip, r-release: disto_0.2.0.zip, r-oldrel: disto_0.2.0.zip |
| OS X binaries: | r-release: disto_0.2.0.tgz, r-oldrel: disto_0.2.0.tgz |
| Old sources: | disto archive |
Please use the canonical form https://CRAN.R-project.org/package=disto to link to this page.


|
|
|
|
In traditional Japanese aesthetics, wabi-sabi is a world view centered on the acceptance of transience and imperfection. The aesthetic is sometimes described as one of beauty that is "imperfect, impermanent, and incomplete.”
| Get Doodles |
In traditional Japanese aesthetics, wabi-sabi is a world view centered on the acceptance of transience and imperfection. The aesthetic is sometimes described as one of beauty that is "imperfect, impermanent, and incomplete.”
| Get Doodles |


{{ charger.name }}, {{ charger.available_stalls }} / {{ charger.total_stalls }} available, {{ charger.distance_miles*1.609344 | number:0 }} km Site closed Navigation request sent. Error sending navigation request.


{% include head.html %}
{% include sidebar.html %}
OSCAR Examples
This page contains example notebooks from the OSCAR project.
{{ content }}


Leclerc to take grid penalty in Brazil
(Reuters) - Ferrari’s Charles Leclerc will take a grid penalty at this weekend’s Brazilian Formula One Grand Prix after team boss Mattia Binotto said his car’s power unit was damaged and needed replacing.
The 22-year-old Monegasque is third in the championship, 14 points ahead of Red Bull’s Max Verstappen with two races remaining.
“Charles will get a new Power Unit (at Interlagos), as his was damaged on Saturday in Austin, which meant he had to use a previously used and less powerful version for the rest of the weekend,” Binotto said in a race preview.
“Fitting a new power unit means taking a grid penalty in Sao Paulo, but we should be back to our normal performance level and show some fighting spirit to finish the season on a high note.”
Mercedes have already won both championships for the sixth year in a row, with Lewis Hamilton sealing his sixth drivers’ title at the U.S. Grand Prix in Texas on Nov. 3.
Leclerc finished fourth in Austin after the engine failure in qualifying, but still set the fastest lap, with team mate Sebastian Vettel retiring from the race with a broken rear right suspension.


Yagna Infrastructure Pvt. Ltd. is a company for all kinds of construction works as well as We also supply all types of quality approved construction materials like Aggregate, Sand and Natural G.S.B. We are primarily based in the Anand district of Gujarat, India.
Owner's Vision
Yagna Infrastructure Pvt. Ltd. is a leader in providing value-added construction services to our customers by creating a successful partnership with them throughout the construction process. Our pledge is to establish lasting relationships with our customers by exceeding their expectations and gaining their trust through exceptional performance by every member of the construction team.


Machine learning method specifically designed for pre-miRNA prediction. It takes advantage of unlabeled sequences to improve the prediction rates even when there are just a few positive examples, when the negative examples are unreliable or are not good representatives of its class. Furthermore, the method can automatically search for negative examples if the user is unable to provide them. MiRNAss can find a good boundary to divide the pre-miRNAs from other groups of sequences; it automatically optimizes the threshold that defines the classes boundaries, and thus, it is robust to high class imbalance. Each step of the method is scalable and can handle large volumes of data.
| Version: | 1.4 |
| Imports: | Matrix, stats, Rcpp, CORElearn, RSpectra |
| LinkingTo: | Rcpp |
| Published: | 2018-01-03 |
| Author: | Cristian Yones |
| Maintainer: | Cristian Yones <cyones at sinc.unl.edu.ar> |
| License: | Apache License 2.0 |
| NeedsCompilation: | yes |
| Materials: | README NEWS |
| CRAN checks: | miRNAss results |
| Reference manual: | miRNAss.pdf |
| Vignettes: |
miRNAss usage |
| Package source: | miRNAss_1.4.tar.gz |
| Windows binaries: | r-devel: miRNAss_1.4.zip, r-release: miRNAss_1.4.zip, r-oldrel: miRNAss_1.4.zip |
| OS X binaries: | r-release: miRNAss_1.4.tgz, r-oldrel: miRNAss_1.4.tgz |
| Old sources: | miRNAss archive |
Please use the canonical form https://CRAN.R-project.org/package=miRNAss to link to this page.


COVIDTracker
Contact
COVID-19 INDIA TRACKER
CONFIRMED
ACTIVE
RECOVERED
DEATHS
DATA VISUALIZED
REGION-WISE TALLY
Region
Confirmed Cases
Active Cases
Recovered
Deaths


0x5a08a4110283E4D8F2adDDaC603CC4f06397b5D8
Please donate to me on the Ethereum Mainnet in ETH, USDT, USDC, BAT or DAI. Thanks!


--- layout: page title: Lean Prover Zulip Chat Archive permalink: archive/144837PRreviews/443021088countingmeasure.html ---
Some terms take a rather long time to elaborate; I suspect it is because of the if then else
using the classical decidability
instance
but I don't know why those terms should take a long time to elaborate at all
{% endraw %}


Full Screen Navigation
Full-Screen Hidden Navigation
It works on mobile devices as well.
Navigation
Link 1
Link 2
Link 3
Link 4


Last updated: 2018-01-08
Code version: ca6e6f8
Tutor: Rebecca Stellato
Tests
Confidence intervals
Assuming:
\[X \sim Bin(n, \pi)\]
\[P(X = x) = {{n}\choose{x}}p^{x}(1-p)^{n-x}\]
With
\[{{n}\choose{x}} = \frac{n!}{x!(n-x)!}\]
Binomial distrubution is approximated by normal distribution with \[\mu = np\] \[\sigma_2 = np*(p-1)\]
\[H_0 = p = p_0\] For a given value, e.g. $0.5%
Note that the the distribution is only symmetric for \(p_0 = 0.5\).
P-value is directly calculated from the distribution.
\[\hat{p} \pm Z_{\alpha(2)}\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}\] Has bad coverage
Add 2 successess and 2 failures (for 95%-confidence interval)
\(r' = r + \frac{Z_{\alpha(2)}^2}{2}\); \(n' = n + Z_{\alpha(2)}^2\); \(\hat{p}' = r'/n'\)
This moves the estimate a little closer to 0.5. Most useful for extreme p-values.
Has better coverage. Report \(p\) and the confidence interval based on \(r'\) and \(n'\).
consider some simulation examples
| . | test B + | test B - |
|---|---|---|
| test A + | 20 | 12 |
| test A - | 2 | 16 |
Test only the discordant paires. They should be equal.
\(H_0\): discordant pairs are equal
\(p = A_+ / B_-\) or \(p = A_- / B_+\) Sample size is sum of discordant pairs Use binomial distribution with 2-sided p-value
Concordant pairs do not matter, all are discarded.
Confidence interval has little meaning. Check McNemar Odds-ratio, it has a confidence interval.
| Drug | Case | Control |
|---|---|---|
| used | 71 | 208 |
| no use | 84 | 173 |
Difference in proportions: \(71/(208+71)\), \(84/(173+84)\)
\[\hat{p_1}-\hat{p_2} \pm Z_{\alpha(2)}\sqrt{\frac{\hat{p_1}(1-\hat{p_1})}{n_1} + \frac{\hat{p_2}(1-\hat{p_2})}{n_2}}\]
Add 1 success and failure in each group. \(r_i' = r_i + \frac{Z_\alpha(2)^2}{4}\); \(n_i' = n_i + \frac{Z_{\alpha(2)}^2}{2}\)
Is best way. Single proportion: binom::binom.confint(..., method = "wilson")
Difference between proportions: Epi::ci.pd(..., method = "Nc")
Test association / difference in a contingency table (2x2 or IxJ)
| Drug | Case | Control |
|---|---|---|
| never use | 71 | 208 |
| past use | 22 | 53 |
| current oral use | 32 | 27 |
| current transdermal use | 30 | 93 |
Null hypothesis is statistical independence:
\[P(A = a_i \& B = b_i) = P(A=a_i)*P(B=b_i)\]
\[E_{ij} = \sum_{j}{O_{ij}} * \sum_{i}{O_{ij}} / n\]
\[T = \sum_{i=1}^I\sum_{j=1}^J\frac{(O_{ij}-E_{ij})^2}{E_ij}\] Difference between expected and observed in a cell, divided (‘normalized’) by the total numer of expected
T is asymptotically chi-square distributed with \((I-1)*(J-1)\). E.g. in a 2x2 table, given marginal probabilities, filling in 1 cell will fix all other cells. E.g. in a 3x2 table, filling in 2 cells will fix all other cells E.g. in a 3x3 table, filling in 4 cells will fix all other cells
It is a two-sided test, although you are always on one side of the distribution.
my_chisq_function <- function(df) {
f = function(x) pchisq(q = x, df = df, lower.tail = T)
return(f)
}
my_chisq <- my_chisq_function(df = 3)
curve(my_chisq, xlim = c(1,30))
Rejecting the null hypothesis does not tell you which ratio is actually different.
You can make a series of 2x2 tables as a ‘post-hoc’ analysis. Usually it does not happen. There is no standard way to do this.
Theory: all expected values \(E_{ij} \geq 5\) Practice: 75% of the \(E_{ij}\) should be larger than 5 Otherwise use Fisher exact test
set.seed(2)
n = 15
p_flip = 0.2
myData <- data.frame(
id = 1:n,
a = sample(c(T,F), replace = T, size = n)
)
myData$b <- myData$a
flippers <- sample(1:n, size = p_flip*n, replace = F)
myData$b[flippers] <- !myData$b[flippers]
myData
id a b
1 1 TRUE TRUE
2 2 FALSE FALSE
3 3 FALSE TRUE
4 4 TRUE TRUE
5 5 FALSE FALSE
6 6 FALSE FALSE
7 7 TRUE TRUE
8 8 FALSE FALSE
9 9 TRUE TRUE
10 10 FALSE FALSE
11 11 FALSE FALSE
12 12 TRUE TRUE
13 13 FALSE TRUE
14 14 TRUE FALSE
15 15 TRUE TRUE
xtabs(~a+b, data = myData)
b
a FALSE TRUE
FALSE 6 2
TRUE 1 6
chisq.test(xtabs(~a+b, data = myData))
Warning in chisq.test(xtabs(~a + b, data = myData)): Chi-squared
approximation may be incorrect
Pearson's Chi-squared test with Yates' continuity correction
data: xtabs(~a + b, data = myData)
X-squared = 3.359, df = 1, p-value = 0.06684
fisher.test(xtabs(~a+b, data = myData))
Fisher's Exact Test for Count Data
data: xtabs(~a + b, data = myData)
p-value = 0.04056
alternative hypothesis: true odds ratio is not equal to 1
95 percent confidence interval:
0.9040436<PHONE_NUMBER>
sample estimates:
odds ratio
13.95942
For contingency tables with low counts
Chi-square test will give statistical significance of rate differences
Odds ratios and relative risks will provide estimates of the effect size.
\[n \geq \frac{(Z_{\alpha}\sqrt{2\bar{p}(1-\bar{p})} + Z_{\beta}\sqrt{p_1(1-p_1)+p_2(1-p_2)})^2}{\delta^2}\]
With \(\bar{p} = \frac{p_1+p_2}{2}\); \(\delta = p_2 - p_1\);
Note: \(n\) is an underestimation of the number needed.
There is a way to do this for paired data (McNemar).
\[n = f_{11} + f_{12} + f_{21} + f_{22}\] \[p = min(\frac{f_{12}}{n}, \frac{f_{21}}{n})\] \[\Psi = max(\frac{f_{12}}{f_{21}}, \frac{f_{21}}{f_{12}})\]
\[n \geq \frac{(Z_{\alpha}\sqrt{\Psi + 1} + Z_{\beta(1)}\sqrt{(\Psi+1)-p(\Psi-1)})^2}{p(\Psi-1)^2}\]
sessionInfo()
R version 3.3.2 (2016-10-31)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: macOS Sierra 10.12.6
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] backports_1.1.0 magrittr_1.5 rprojroot_1.2 tools_3.3.2
[5] htmltools_0.3.6 yaml_2.1.16 Rcpp_0.12.14 stringi_1.1.6
[9] rmarkdown_1.8 knitr_1.18 git2r_0.20.0 stringr_1.2.0
[13] digest_0.6.13 evaluate_0.10.1
This R Markdown site was created with workflowr


| Date | Editor | Edit Type | Description |
|---|---|---|---|
|
|
|
|
|
In most cases, a mint should point to a parent region. In certain cases of individual, definable workshops, the broader concept can be the primary mint for a place, within the same general field of numismatics.
Search for an existing ID to forward. By default, this ID is usually the same RDF class.


Files | |
| file | CDepthBuffer.cpp |
| file | CDepthBuffer.h [code] |
| file | CDrawContext.cpp |
| file | CDrawContext.h [code] |
| file | CFrameBuffer.cpp |
| file | CFrameBuffer.h [code] |
| file | CGeometryShader.h [code] |
| file | CGraphicsContext.cpp |
| file | CGraphicsContext.h [code] |
| file | CIndexBuffer.cpp |
| file | CIndexBuffer.h [code] |
| file | COpenGLImplementation.cpp |
| file | COpenGLImplementation.h [code] |
| file | CPipelineState.cpp |
| file | CPipelineState.h [code] |
| file | CPixelShader.h [code] |
| file | CRenderTarget.cpp |
| file | CRenderTarget.h [code] |
| file | CShaderProgram.cpp |
| file | CShaderProgram.h [code] |
| file | CTexture.cpp |
| file | CTexture.h [code] |
| file | CVertexBuffer.cpp |
| file | CVertexBuffer.h [code] |
| file | CVertexShader.h [code] |
| file | glad.cpp |
| file | ionGraphicsGL.h [code] |
| file | Utilities.cpp |
| file | Utilities.h [code] |
Generated on Sun May 6 2018 for ionEngine v0.4.5


You are viewing an historical archive of past issues. Please
report new issues to the appropriate project issue tracker on
GitHub.
Comments
-
Greg Hurrell
Thanks for the submission. I'll have to look and see whether iTunes' AppleScript dictionary includes any kind of hook that could be used to instruct it to eject the iPod...
-
Greg Hurrell
Yeah, looks like there is something in there:
eject v : eject the specified iPod eject [source] : the iPod to eject source n [inh. item] : a music source (music library, CD, device, etc.) elements contains audio CD playlists, device playlists, library playlists, playlists, radio tuner playlists, user playlists; contained by application. properties capacity (double integer, r/o) : the total size of the source if it has a fixed size free space (double integer, r/o) : the free space on the source if it has a fixed size kind (library/iPod/audio CD/MP3 CD/device/radio tuner/shared library/unknown, r/o)
-
Greg Hurrell
Status changed:
- From: New
- To: Open
Add a comment
Comments are now closed for this issue.


~> xiaq@macarch LOCATION 103 ~/go/src/github.com/elves/elvish/website 49 ~/go/src/github.com/elves/elvish/website/learn │ 47 ~/go/src/github.com/elves/elvish │ 27 ~/go/src/github.com/elves/elvish/pkg │


--- layout: post title: How to ignore files only locally in git? date: 2017-06-02 08:27:15.000000000 +02:00 type: post parent_id: '0' published: true password: '' status: publish tags: - Linux/Ubuntu - Tricks - Git meta: _publicize_done_external: a:1:{s:7:"twitter";a:1:{i:16076855;s:56:"https://twitter.com/MphoMphego/status/870527238856261632";}} publicize_google_plus_url: https://plus.google.com/+MphoMphego/posts/iPLgiohrGJi _rest_api_published: '1' _rest_api_client_id: "-1" _wpas_skip_16076853: '1' _publicize_job_id: '5692894808' publicize_linkedin_url: https://www.linkedin.com/updates?discuss=&scope=97183493&stype=M&topic=6276292925714501633&type=U&a=v2a3 _publicize_done_16016023: '1' _wpas_done_15845576: '1' _publicize_done_16257272: '1' _wpas_done_16076855: '1' publicize_twitter_user: MphoMphego _publicize_done_16966120: '1' _wpas_done_16750486: '1' author: login: mpho112 email<EMAIL_ADDRESS>display_name: Mpho first_name: '' last_name: '' ---
I have automated tests running everyday, and the issue is that those tests make necessary changes to certain files depending on which test is running. This becomes a problem when I have to report the changes in the repo, as this will report untracked changed, which means I have to ignore certain files such that local Git doesn't report changes in them specifically.
By making sure to run git update-index --assume-unchanged ./matplotlibrc
after making the addition to the exclude file this means changes won't be picked up until then.
git update-index --assume-unchanged index.rst
git update-index --assume-unchanged matplotlibrc
Before:
After:
Changes made but untracked, nice!!!


Transition Properties
| Type | Description |
|---|---|
| fade | Transition opacity from 1 to 0. |
| scale | Transition scale from 1 to 0. |
| up | Transition element from down to up. |
| right | Transition element from left to right. |
| down | Transition element from up to down. |
| left | Transition element from up to down. |
| rotate-y | Transition rotation along the y axis. |
| rotate-x | Transition rotation along the x axis. |

This is a cool feature about your product! It really separates you from the crowd.

This is a cool feature about your product! It really separates you from the crowd.

This is a cool feature about your product! It really separates you from the crowd.

This is a cool feature about your product! It really separates you from the crowd.

This is a cool feature about your product! It really separates you from the crowd.

This is a cool feature about your product! It really separates you from the crowd.

This is a cool feature about your product! It really separates you from the crowd.

This is a cool feature about your product! It really separates you from the crowd.

Transition Types
| Type | Description |
|---|---|
| in-out | This means the element starts in the out state and tweens to the in state. |
| out-in | This means the element starts in the in state and tweens to the out state. |
| in | This means the element starts in the in state and tweens to the out state, but only once. After the scroll is fully triggered, the element will stay in the out state. |
| out | This means the element starts in the out state and tweens to the in state, but only once. After the scroll is fully triggered, the element will stay in the in state. |

This is a cool feature about your product! It really separates you from the crowd.

This is a cool feature about your product! It really separates you from the crowd.

This is a cool feature about your product! It really separates you from the crowd.

This is a cool feature about your product! It really separates you from the crowd.

Staggered Transitions

This is a cool feature about your product! It really separates you from the crowd.

This is a cool feature about your product! It really separates you from the crowd.

This is a cool feature about your product! It really separates you from the crowd.

This is a cool feature about your product! It really separates you from the crowd.

This is a cool feature about your product! It really separates you from the crowd.

This is a cool feature about your product! It really separates you from the crowd.


Dean Wampler, Ph.D.
Typesafe
<EMAIL_ADDRESS>@deanwampler
This workshop focuses on the new Spark SQL API, which integrates structured data and embedded SQL queries with normal Spark code. It even has a Hive integration module so you can query existing Hive tables, even create and delete them.
We pick up where my Apache Spark: An Introductory Workshop left off, including the numbering used for the examples.
I'll assume you've already worked through that workshop and not repeat details about Spark basics.
TBD. For now, see the Spark SQL description.
If you're using the Typesafe Activator UI, search for activator-spark
and install it in the UI. The code is built automatically.
If you grabbed this workshop from Github, you'll need to install sbt
and use a command line to build and run the applications. In that case, see the sbt website for instructions on installing sbt
.
To compile the code and run the tests, either use the Activator UI test link or start sbt
and enter test
at the prompt.
Either way, the code is compiled and the tests are executed. They should pass without error. Note that tests are provided for all the exercises, except the Spark Streaming exercise, which uses multiple processes.
Next, let's try running one of the exercises.
In Activator, use the run panel, select the first exercise, SparkSQL9
, in the bullet items under "Main Class", and click the "Start" button. The "Logs" panel shows some information. Note the output
directories listed in the output. Use a file browser to find those directories to view the output written in those locations.
In sbt
, enter run
. It will present the same list of main classes. Enter the number for SparkSQL9
to run it.
Note that the some exercises have package names with the word solns
. They are solutions to the exercises.
Here is a list of the exercises. In subsequent sections, we'll dive into the details for each one. Note that each name ends with a number, indicating the order in which we'll discuss and try them:
data
directory has a README that discusses the sources of the data files.) This script ends by writing data in the de-facto standard Parquet format that is increasingly popular in Big Data applications.Let's now work through these exercises...
TODO
Reads and parses the King James Bible text, kjvdat.txt
.
As in the previous Spark Workshop, command line options can be used to override the defaults. We'll have to use sbt
from a command window to use this feature (rather than the Activator UI), and we'll have to use the run-main
task, which lets us specify a particular main
to run and optional arguments.
The "\" characters in the following and subsequent command descriptions are used to indicate long lines that I wrapped to fit. Enter the commands on a single line without the "\". I use [...]
to indicate optional arguments, and |
to indicate alternative flags:
run-main spark.SparkSQL9 [ -h | --help] \
[-i | --in | --inpath input] \
[-o | --out | --outpath output] \
[-m | --master master] \
[-q | --quiet]
Where the options have the following meanings:
-h | --help Show help and exit.
-i ... input Read this input source (default: data/kjvdat.txt).
-o ... output Write to this output location (default: output/verses.parquet).
-m ... master local, local[k], etc. as discussed previously.
-q | --quiet Suppress some informational output.
The options work the same as before, except we don't append a timestamp to the output path, because its contents will be read by the next exercise.
NOTE: Because we don't append a timestamp to the output path, you'll get a runtime error if the directory already exists. Just delete or rename the existing directory, or specify a different output path.
Code description TODO.
As before, there are suggested exercises at the end of the source file. Some solutions are provided in the solns
source package.
TODO
Reads the Parquet output of the previous example and works with the data using the SQL API.
The command-line options are similar, but there is no output option as all results are printed to the console:
run-main spark.SparkSQLParquet10 [ -h | --help] \
[-i | --in | --inpath input] \
[-m | --master master] \
[-q | --quiet]
Code description TODO.
TODO
This is actually a script file, but you'll find it most useful if you start the console
in SBT, then copy and paste the statements, to see what each one does. Then you can experiment with the API, especially if you already know Hive!.
Note that the Hive "metadata" is stored in a megastore
directory created in the current working directory. This is written and managed by Hive's embedded Derby SQL store, but is not a production configuration.
Code description TODO.
This template is not a complete Apache Spark SQL tutorial and the API is evolving rapidly. For example, built-in support for JSON-formatted data is being added in a subsequent release. To learn more, see the following:


---
layout: post
title: ! 'Software Development: Where are you now?'
tags:
- Application Lifecycle Management
- Project Management
- Software Configuration Management
- Software Development
- Work Practices
status: publish
type: post
published: true
meta: {}
---
When you start to think about a software project one of the things you absolutely need to know is "Where are we now?" (along with "Where do we want to go?" and "How are we going to get there?"). This question is usually not thought about that hard, and if the team have been working together for some time they generally "just know" anyway. However for an ousider it's an important question to think about. I generally look for the following:
- What version control tools are in use and how is the information set up and used
- Is there any documentation of process and
- Does anyone follow the process documentation
- How do the customers feel about the work the team does - how do they think the team should change (that can be a hard question to get honest answers for)
- How are requests and issues documented (e.g. emails, post-it notes, bug tracking tool ...) and how is the information used
- What is the technology used (e.g. frameworks, languages, build tools, IDEs, deployment ...)
- What are the current 'pain' points (e.g. too much rework, too hard to understand code base, can't refactor etc.)
- How much budget and autonomy does the team have
This will help you build an informal map to assist in in planning any work and identifying issues that may need addressing outside of the current project work. The information should be maintained in some easy to understand form e.g. a Mindmap or WiKi and updated every month or two.