| Jump to DART Documentation Main Index |
Main program for evaluating filter performance in observation space. Primarily, the prior or posterior ensemble mean (and spread) are compared to the observation and several quantities are calculated. These quantities are then saved in a netCDF file that has all the metadata to create meaningful figures.
Each obs_seq.final file contains an observation sequence that has multiple 'copies' of the observation. One copy is the actual observation, another copy is the prior ensemble mean estimate of the observation, one is the spread of the prior ensemble estimate, one may be the prior estimate from ensemble member 1, ... etc. If the original observation sequence is the result of a 'perfect model' experiment, there is an additional copy called the 'truth' - the noise-free expected observation given the true model state. Since this copy does not, in general, exist for the high-order models, all comparisons are made with the copy labelled 'observation'. There is also a namelist variable (use_zero_error_obs) to compare against the 'truth' instead; the observation error variance is then automatically set to zero.
Each ensemble member applies a forward observation operator to the state to compute
the "expected" value of an observation. Please note:
the forward observation operator is applied AFTER any prior inflation
has taken place! Similarly, the forward observation operator is applied AFTER any posterior
inflation. This has always been the case. For a detailed
look at the relationship between the observation operators and inflation, please look at the
Detailed Program Execution Flow
section of filter.html.
Given multiple estimates of
the observation, several quantities can be calculated. It is possible to
compute the expected observations from the state vector before
assimilating (the "guess", "forecast", or "prior") or after the
assimilation (the "analysis", or "posterior").
Even with input.nml:filter_nml:num_output_obs_members set to 0; the full [prior,posterior] ensemble mean and [prior,posterior] ensemble spread are preserved in the obs_seq.final file. Consequently, the ensemble means and spreads are used to calculate the diagnostics. If the input.nml:filter_nml:num_output_obs_members is set to 80 (for example); the first 80 ensemble members prior and posterior "expected" values of the observation are also included. In this case, the obs_seq.final file contains enough information to calculate a rank histograms, verify forecasts, etc. The ensemble means are still used for many other calculations.
Since this program is fundamentally interested in the response as a function of region, there are three versions of this program; one for each of the oned, threed_sphere, or threed_cartesian location modules (location_mod.f90). It did not make sense to ask the lorenz_96 model what part of North America you'd like to investigate or how you would like to bin in the vertical. The low-order models write out similar netCDF files and the Matlab scripts have been updated accordingly. The oned observations have locations conceptualized as being on a unit circle, so only the namelist input variables pertaining to longitude are used.
Identity observations (only possible from "perfect model experiments") are already explored with state-space diagnostics, so obs_diag simply skips them.
obs_diag is designed to explore the effect of the assimilation in three ways; 1) as a function of time for a particular variable and level (this is the figure on the left), 2) as a time-averaged vertical profile (figure in the middle), and sometimes 3) in terms of a rank histogram - "Where does the actual observation rank relative to the rest of the ensemble?" (figures on the right). The figures on the left and center were created by several Matlab® scripts that query the obs_diag_output.nc file: DART/diagnostics/matlab/plot_evolution.m and plot_profile.m. Both of these takes as input a file name and a 'quantity' to plot ('rmse','spread','totalspread', ...) and exhaustively plots the quantity (for every variable, every level, every region) in a single matlab figure window - and creates a series of .ps files with multiple pages for each of the figures. The directory gets cluttered with them. The rank histogram information can easily be plotted with ncview, a free third-party piece of software or with plot_rank_histogram.m.
obs_diag can be configured to compare the ensemble estimates against the 'observation' copy or the 'truth' copy based on the setting of the use_zero_error_obs namelist variable.
The observation sequence files contain only the time of the observation, nothing of the assimilation interval, etc. - so it requires user guidance to declare what sort of temporal binning for the temporal evolution plots. I do a 'bunch' of arithmetic on the namelist times to convert them to a series of temporal bin edges that are used when traversing the observation sequence. The actual algorithm is that the user input for the start date and bin width set up a sequence that ends in one of two ways ... the last time is reached or the number of bins has been reached.
obs_diag reads obs_seq.final files and calculates the following quantities (in no particular order) for an arbitrary number of regions and levels. obs_diag creates a netCDF file called obs_diag_output.nc. It is necessary to query the CopyMetaData variable to determine the storage order (i.e. "which copy is what?") if you want to use your own plotting routines.
| Nposs | The number of observations available to be assimilated. |
| Nused | The number of observations that were assimilated. |
| NbadUV | the number of velocity observations that had a matching component that was not assimilated; |
| NbadLV | the number of observations that were above or below the highest or lowest model level, respectively; |
| rmse | The root-mean-squared error (the horizontal wind components are also used to calculate the vector wind velocity and its RMS error). |
| bias | The simple sum of forecast - observation. The bias of the horizontal wind speed (not velocity) is also computed. |
| spread | The standard deviation of the univariate obs. DART does not exploit the bivariate nature of U,V winds and so the spread of the horizontal wind is defined as the sum of the spreads of the U and V components. |
| totalspread | The total standard deviation of the estimate. We pool the ensemble variance of the observation plus the observation error variance and take the square root. |
| NbadDARTQC | the number of observations that had a DART QC value (> 1 for a prior, > 3 for a posterior) |
| observation | the mean of the observation values |
| ens_mean | the ensemble mean of the model estimates of the observation values |
| N_trusted | the number of implicitly trusted observations, regardless of DART QC |
| N_DARTqc_0 | the number of observations that had a DART QC value of 0 |
| N_DARTqc_1 | the number of observations that had a DART QC value of 1 |
| N_DARTqc_2 | the number of observations that had a DART QC value of 2 |
| N_DARTqc_3 | the number of observations that had a DART QC value of 3 |
| N_DARTqc_4 | the number of observations that had a DART QC value of 4 |
| N_DARTqc_5 | the number of observations that had a DART QC value of 5 |
| N_DARTqc_6 | the number of observations that had a DART QC value of 6 |
| N_DARTqc_7 | the number of observations that had a DART QC value of 7 |
| N_DARTqc_8 | the number of observations that had a DART QC value of 8 |
The temporal evolution of the above quantities for every observation type (RADIOSONDE_U_WIND_COMPONENT, AIRCRAFT_SPECIFIC_HUMIDITY, ...) is recorded in the output netCDF file - obs_diag_output.nc. This netCDF file can then be loaded and displayed using the Matlab® scripts in ..../DART/diagnostics/matlab. (which may depend on functions in ..../DART/matlab). The temporal, geographic, and vertical binning are under namelist control. Temporal averages of the above quantities are also stored in the netCDF file. Normally, it is useful to skip the 'burn-in' period - the amount of time to skip is under namelist control.
The DART QC flag is intended to provide information about whether the observation was assimilated, evaluated only, whether the assimilation resulted in a 'good' observation, etc. DART QC values lower than 2 indicate the prior and posteriors are OK. DART QC values higher than 3 were not assimilated or evaluated. Here is the table that should explain things more fully:
| DART QC flag value | meaning |
|---|---|
| 0 | observation assimilated |
| 1 | observation evaluated only (because of namelist settings) |
| 2 | assimilated, but the posterior forward operator failed |
| 3 | evaluated only, but the posterior forward operator failed |
| 4 | prior forward operator failed |
| 5 | not used because observation type not listed in namelist |
| 6 | rejected because incoming observation QC too large |
| 7 | rejected because of a failed outlier threshold test |
| 8 | vertical conversion failed |
| 9+ | reserved for future use |
obs_diag has several improvements:
This namelist is read from the file input.nml. Namelists start with an ampersand '&' and terminate with a slash '/'. Character strings that contain a '/' must be enclosed in quotes to prevent them from prematurely terminating the namelist.
&obs_diag_nml obs_sequence_name = '' obs_sequence_list = '' first_bin_center = 2003, 1, 1, 0, 0, 0 last_bin_center = 2003, 1, 2, 0, 0, 0 bin_separation = 0, 0, 0, 6, 0, 0 bin_width = 0, 0, 0, 6, 0, 0 time_to_skip = 0, 0, 1, 0, 0, 0 max_num_bins = 1000 plevel = -888888.0 hlevel = -888888.0 mlevel = -888888 plevel_edges = -888888.0 hlevel_edges = -888888.0 mlevel_edges = -888888 Nregions = 0 lonlim1 = -888888.0 lonlim2 = -888888.0 latlim1 = -888888.0 latlim2 = -888888.0 reg_names = 'null' trusted_obs = 'null' create_rank_histogram = .true. outliers_in_histogram = .false. use_zero_error_obs = .false. verbose = .false. /
These values may be more useful:
obs_sequence_list = 'input_file_list.txt' plevel = 1000, 925, 850, 700, 500, 400, 300, 250, 200, 150, 100, -or- plevel_edges = 1050, 962.5, 887.5, 775, 600, 450, 350, 275, 225, 175, 125, 75 hlevel = 1000, 2000, 3000, 4000, 5000, 6000, 7000, 8000, 9000, 10000, 11000, -or- hlevel_edges = 0, 1500, 2500, 3500, 4500, 5500, 6500, mlevel = 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, -or- mlevel_edges = 0.5, 1.5, 2.5, 3.5, 10.5, Nregions = 4 lonlim1 = 0.0, 0.0, 0.0, 235.0 lonlim2 = 360.0, 360.0, 360.0, 295.0 latlim1 = 20.0, -80.0, -20.0, 25.0 latlim2 = 80.0, -20.0, 20.0, 55.0 reg_names = 'Northern Hemisphere', 'Southern Hemisphere', 'Tropics', 'North America'
The date-time integer arrays in this namelist have the form
(YYYY, MM, DY, HR, MIN, SEC).
The allowable ranges for the region
boundaries are: latitude [-90.,90], longitude [0.,Inf.]
You can only specify either obs_sequence_name or obs_sequence_list -- not both. One of them has to be an empty string ... i.e. ''.
| Item | Type | Description |
|---|---|---|
| obs_sequence_name | character(len=256), dimension(100) | An array of names of observation sequence files. These may be relative or absolute filenames. If this is set, obs_sequence_list must be set to ' ' (empty string). |
| obs_sequence_list | character(len=256) | Name of an ascii text file which contains a list of one or more observation sequence files, one per line. If this is specified, obs_sequence_name must be set to ' '. Can be created by any method, including sending the output of the 'ls' command to a file, a text editor, or another program. If this is set, obs_sequence_name must be set to ' ' (empty string). |
| first_bin_center | integer, dimension(6) | first timeslot of the first obs_seq.final file to process. The six integers are: year, month, day, hour, hour, minute, second, in that order. obs_diag has improved run-time output that reports the time and date of the first and last observations in every observation sequence file. Look for the string 'First observation date' in the logfile. If the verbose is 'true', it is also written to the screen. |
| last_bin_center | integer, dimension(6) | last timeslot of interest. (reminder: the last timeslot of day 1 is hour 0 of day 2) The six integers are: year, month, day, hour, hour, minute, second, in that order. This does not need to be exact, the values from first_bin_center and bin_separation are used to populate the time array and stop on or before the time defined by last_bin_center. See also max_num_bins. |
| bin_separation | integer, dimension(6) | Time between bin centers. The year and month values must be zero. |
| bin_width | integer, dimension(6) | Time span around bin centers in which obs will be compared. The year and month values must be zero. Frequently, but not required to be, the same as the values for bin_separation. 0 |
| time_to_skip | integer, dimension(6) | Time span at the beginning to skip when calculating vertical profiles of rms error and bias. The year and month values must be zero. Useful because it takes some time for the assimilation to settle down from the climatological spread at the start. time_to_skip is an amount of time AFTER the first edge of the first bin. |
| max_num_bins | integer | This provides an alternative way to declare the last_bin_center. If max_num_bins is set to '10', only 10 timesteps will be output - provided last_bin_center is set to some later date. |
| plevel | real, dimension(50) | The midpoints defining the pressure levels for the vertical binning. There is no specification of bin width - a continuum is used. If a single midpoint value is entered, the bin edges are +/- 10% of the midpoint value. If you'd like to change that see the routine Rmidpoints2edges(). You may specify either plevel or plevel_edges, but not both. |
| plevel_edges | real, dimension(51) | The edges defining the pressure levels for the vertical binning. You may specify either plevel or plevel_edges, but not both. |
| hlevel | real, dimension(50) | Same, but for observations that have height(m) or depth(m) as the vertical coordinate. |
| hlevel_edges | real, dimension(51) | The edges defining the height (or depth) levels for the vertical binning. You may specify either hlevel or hlevel_edges, but not both. |
| mlevel | real, dimension(50) | Same, but for observations that have model level as the vertical coordinate. |
| mlevel_edges | real, dimension(51) | The edges defining the model levels for the vertical binning. You may specify either mlevel or mlevel_edges, but not both. |
| Nregions | integer | Number of regions of the globe for which obs space diagnostics are computed separately. Must be between [1,50]. If 50 is not enough, increase obs_diag.f90MaxRegions and recompile. |
| lonlim1 | real, dimension(50) | Westernmost longitudes of each of the regions. |
| lonlim2 | real, dimension(50) | Easternmost longitudes of each of the regions. If any of these values is less than the westernmost values, it defines a region that spans the prime meridian. e.g. a specification of lonlim1 = 330 , lonlim2 = 50 could identify a region like "Africa". |
| latlim1 | real, dimension(50) | Southernmost latitudes of the regions. |
| latlim2 | real, dimension(50) | Northernmost latitudes of the regions. |
| reg_names | character(len=129), dimension(50) | Array of names for the regions to be analyzed. Will be used for plot titles. |
| trusted_obs | character(len=32), dimension(50) | list of observation types that must participate in the calculation of the statistics, regardless of the DART QC (provided that the forward observation operator can still be applied without failure). e.g. 'RADIOSONDE_TEMPERATURE', ... For more details, see the section on Trusted observations. |
| use_zero_error_obs | logical | if .true., the observation copy used for the statistics calculations will be 'truth'. Only 'perfect' observations (from perfect_model_obs) have this copy. The observation error variance will be set to zero. |
| create_rank_histogram | logical | if .true. and there are actual ensemble estimates of the observations in the obs_seq.final (i.e. filter_nml:num_output_obs_members is larger than zero), a rank histogram will be created. |
| outliers_in_histogram | logical | if .true. the observations that have been rejected by the outlier threshhold mechanism will be included in the calculation of the rank histogram. |
| verbose | logical | switch controlling amount of run-time output. |
obs_sequence_mod obs_kind_mod obs_def_mod (and possibly other obs_def_xxx mods) assim_model_mod random_seq_mod model_mod location_mod types_mod time_manager_mod utilities_mod sort_mod
Obs_diag may require a model input file from which to get grid information, metadata, and links to modules providing the models expected observations. It all depends on what's needed by the model_mod.f90
Every observation type encountered in the observation sequence file is tracked separately, and aggregated into temporal and 3D spatial bins. There are two main efforts to this program. One is to track the temporal evolution of any of the quantities available in the netCDF file for any possible observation type:
The other is to explore the vertical profile of a particular observation kind. By default, each observation kind has a 'guess/prior' value and an 'analysis/posterior' value - which shed some insight into the innovations.
The obs_diag_output.nc output file has all the metadata I could think of, as well as separate variables for every observation type in the observation sequence file. Furthermore, there is a separate variable for the 'guess/prior' and 'analysis/posterior' estimate of the observation. To distinguish between the two, a suffix is appended to the variable name. An example seems appropriate:
... char CopyMetaData(copy, stringlength) ; CopyMetaData:long_name = "quantity names" ; char ObservationTypes(obstypes, stringlength) ; ObservationTypes:long_name = "DART observation types" ; ObservationTypes:comment = "table relating integer to observation type string" ; float RADIOSONDE_U_WIND_COMPONENT_guess(time, copy, plevel, region) ; RADIOSONDE_U_WIND_COMPONENT_guess:_FillValue = -888888.f ; RADIOSONDE_U_WIND_COMPONENT_guess:missing_value = -888888.f ; float RADIOSONDE_V_WIND_COMPONENT_guess(time, copy, plevel, region) ; RADIOSONDE_V_WIND_COMPONENT_guess:_FillValue = -888888.f ; RADIOSONDE_V_WIND_COMPONENT_guess:missing_value = -888888.f ; ... float MARINE_SFC_ALTIMETER_guess(time, copy, surface, region) ; MARINE_SFC_ALTIMETER_guess:_FillValue = -888888.f ; MARINE_SFC_ALTIMETER_guess:missing_value = -888888.f ; ... float RADIOSONDE_WIND_VELOCITY_guess(time, copy, plevel, region) ; RADIOSONDE_WIND_VELOCITY_guess:_FillValue = -888888.f ; RADIOSONDE_WIND_VELOCITY_guess:missing_value = -888888.f ; ... float RADIOSONDE_U_WIND_COMPONENT_analy(time, copy, plevel, region) ; RADIOSONDE_U_WIND_COMPONENT_analy:_FillValue = -888888.f ; RADIOSONDE_U_WIND_COMPONENT_analy:missing_value = -888888.f ; float RADIOSONDE_V_WIND_COMPONENT_analy(time, copy, plevel, region) ; RADIOSONDE_V_WIND_COMPONENT_analy:_FillValue = -888888.f ; RADIOSONDE_V_WIND_COMPONENT_analy:missing_value = -888888.f ; ...
There are several things to note:
Believe it or not, there are another set of netCDF variables specifically for the vertical profiles, essentially duplicating the previous variables but without the 'time' dimension. These are distinguished by the suffix added to the observation kind - 'VPguess' and 'VPanaly' - 'VP' for Vertical Profile.
... float SAT_WIND_VELOCITY_VPguess(copy, plevel, region) ; SAT_WIND_VELOCITY_VPguess:_FillValue = -888888.f ; SAT_WIND_VELOCITY_VPguess:missing_value = -888888.f ; ... float RADIOSONDE_U_WIND_COMPONENT_VPanaly(copy, plevel, region) ; RADIOSONDE_U_WIND_COMPONENT_VPanaly:_FillValue = -888888.f ; RADIOSONDE_U_WIND_COMPONENT_VPanaly:missing_value = -888888.f ; ...
Observations flagged as 'surface' do not participate in the vertical profiles (Because surface variables cannot exist on any other level, there's not much to plot!). Observations on the lowest level DO participate. There's a difference!
If it is possible to calculate a rank histogram, there will also be :
... int RADIOSONDE_U_WIND_COMPONENT_guess_RankHi(time, rank_bins, plevel, region) ; ... int RADIOSONDE_V_WIND_COMPONENT_guess_RankHi(time, rank_bins, plevel, region) ; ... int MARINE_SFC_ALTIMETER_guess_RankHist(time, rank_bins, surface, region) ; ...
as well as some global attributes. The attributes reflect the namelist settings and can be used by plotting routines to provide additional annotation for the histogram.
:DART_QCs_in_histogram = 0, 1, 2, 3, 7 ; :outliers_in_histogram = "TRUE" ;
Please note:
| Instructions for viewing the rank histogram with ncview. | |
| Instructions for viewing the rank histogram with Matlab. |
This needs to be stated up front: obs_diag is a post-processor; it cannot influence the assimilation. One interpretation of a TRUSTED observation is that the assimilation should always use the observation, even if it is far from the ensemble. At present (23 Feb 2015), the filter in DART does not forcibly assimilate any one observation and selectively assimilate the others. Still, it is useful to explore the results using a set of 'trusted type' observations, whether they were assimilated, evaluated, or rejected by the outlier threshhold. This is the important distinction. The diagnostics can be calculated differently for each observation type.
The normal diagnostics calculate the metrics (rmse, bias, etc.) only for the 'good' observations - those that were assimilated or evaluated. The outlier_threshold essentially defines what observations are considered too far from the ensemble prior to be useful. These observations get a DART QC of 7 and are not assimilated. The observations with a DART QC of 7 do not contribute the the metrics being calculated. Similarly, if the forward observation operator fails, these observations cannot contribute. When the operator fails, the 'expected' observation value is 'MISSING', and there is no ensemble mean or spread.
'Trusted type' observation metrics are calculated using all the observations that were assimilated or evaluated AND the observations that were rejected by the outlier threshhold. obs_diag can post-process the DART QC and calculate the metrics appropriately for observation types listed in the trusted_obs namelist variable. If there are trusted observation types specified for obs_diag, the obs_diag_output.nc has global metadata to indicate that a different set of criteria were used to calculate the metrics. The individual variables also have an extra attribute. In the following output, input.nml:obs_diag_nml:trusted_obs was set: trusted_obs = 'RADIOSONDE_TEMPERATURE', 'RADIOSONDE_U_WIND_COMPONENT'
... float RADIOSONDE_U_WIND_COMPONENT_guess(time, copy, plevel, region) ; RADIOSONDE_U_WIND_COMPONENT_guess:_FillValue = -888888.f ; RADIOSONDE_U_WIND_COMPONENT_guess:missing_value = -888888.f ; RADIOSONDE_U_WIND_COMPONENT_guess:TRUSTED = "TRUE" ; float RADIOSONDE_V_WIND_COMPONENT_guess(time, copy, plevel, region) ; RADIOSONDE_V_WIND_COMPONENT_guess:_FillValue = -888888.f ; RADIOSONDE_V_WIND_COMPONENT_guess:missing_value = -888888.f ; ... // global attributes: ... :trusted_obs_01 = "RADIOSONDE_TEMPERATURE" ; :trusted_obs_02 = "RADIOSONDE_U_WIND_COMPONENT" ; :obs_seq_file_001 = "cam_obs_seq.1978-01-01-00000.final" ; :obs_seq_file_002 = "cam_obs_seq.1978-01-02-00000.final" ; :obs_seq_file_003 = "cam_obs_seq.1978-01-03-00000.final" ; ... :MARINE_SFC_ALTIMETER = 7 ; :LAND_SFC_ALTIMETER = 8 ; :RADIOSONDE_U_WIND_COMPONENT--TRUSTED = 10 ; :RADIOSONDE_V_WIND_COMPONENT = 11 ; :RADIOSONDE_TEMPERATURE--TRUSTED = 14 ; :RADIOSONDE_SPECIFIC_HUMIDITY = 15 ; :AIRCRAFT_U_WIND_COMPONENT = 21 ; ...
There is ONE ambiguous case for trusted observations. There may be instances in which the observation fails the outlier threshhold test (which is based on the prior) and the posterior forward operator fails. DART does not have a QC that explicilty covers this case. The current logic in obs_diag correctly handles these cases except when trying to use 'trusted' observations. There is a section of code in obs_diag that may be enabled if you are encountering this ambiguous case. As obs_diag runs, a warning message is issued and a summary count is printed if the ambiguous case is encountered. What normally happens is that if that specific observation type is trusted, the posterior values include a MISSING value in the calculation which makes them inaccurate. If the block of code is enabled, the DART QC is recast as the PRIOR forward observation operator fails. This is technically incorrect, but for the case of trusted observations, it results in only calculating statistics for trusted observations that have a useful prior and posterior. This should not be used unless you are willing to intentionally disregard 'trusted' observations that were rejected by the outlier threshhold. Since the whole point of a trusted observation is to include observations potentially rejected by the outlier threshhold, you see the problem. Some people like to compare the posteriors. THAT can be the problem.
if ((qc_integer == 7) .and. (abs(posterior_mean(1) - MISSING_R8) < 1.0_r8)) then write(string1,*)'WARNING ambiguous case for obs index ',obsindex string2 = 'obs failed outlier threshhold AND posterior operator failed.' string3 = 'Counting as a Prior QC == 7, Posterior QC == 4.' if (trusted) then ! COMMENT string3 = 'WARNING changing DART QC from 7 to 4' ! COMMENT qc_integer = 4 endif call error_handler(E_MSG,'obs_diag',string1,text2=string2,text3=string3) num_ambiguous = num_ambiguous + 1 endif
obs_diag is built in .../DART/models/your_model/work, in the same way as the other DART components.
There are two ways to specify input files for obs_diag. You can either specify the name of a file containing a list of files (in obs_sequence_list), or you may specify a list of files via obs_sequence_name.
| In this example, we will be accumulating metrics for 30 days over the entire globe. The obs_diag_output.nc file will have exactly ONE timestep in it (so it won't be much use for the plot_evolution functions) - but the plot_profile functions and the plot_rank_histogram function will be used to explore the assimilation. By way of an example, we will NOT be using outlier observations in the rank histogram. Lets presume that all your obs_seq.final files are in alphabetically-nice directories: |
/Exp1/Dir01/obs_seq.final /Exp1/Dir02/obs_seq.final /Exp1/Dir03/obs_seq.final ... /Exp1/Dir99/obs_seq.final
The first step is to create a file containing the list of observation sequence files you want to use. This can be done with the unix command 'ls' with the -1 option (that's a number one) to put one file per line.
It is necessary to turn on the verbose option to check the first/last times that will be used for the histogram. Then, the namelist settings for 2008 07 31 12Z through 2008 08 30 12Z are:
&obs_diag_nml obs_sequence_name = '' obs_sequence_list = 'obs_file_list.txt' first_bin_center = 2008, 8,15,12, 0, 0 last_bin_center = 2008, 8,15,12, 0, 0 bin_separation = 0, 0,30, 0, 0, 0 bin_width = 0, 0,30, 0, 0, 0 time_to_skip = 0, 0, 0, 0, 0, 0 max_num_bins = 1000 Nregions = 1 lonlim1 = 0.0 lonlim2 = 360.0 latlim1 = -90.0 latlim2 = 90.0 reg_names = 'Entire Domain' create_rank_histogram = .true. outliers_in_histogram = .false. verbose = .true. /
then, simply run obs_diag in the usual manner - you may want to save the run-time output to a file. Here is a portion of the run-time output:
... Region 1 Entire Domain (WESN): 0.0000 360.0000 -90.0000 90.0000 Requesting 1 assimilation periods. epoch 1 start day=148865, sec=43201 epoch 1 center day=148880, sec=43200 epoch 1 end day=148895, sec=43200 epoch 1 start 2008 Jul 31 12:00:01 epoch 1 center 2008 Aug 15 12:00:00 epoch 1 end 2008 Aug 30 12:00:00 ... MARINE_SFC_HORIZONTAL_WIND_guess_RankHis has 0 "rank"able observations. SAT_HORIZONTAL_WIND_guess_RankHist has 0 "rank"able observations. ...
Discussion: It should be pretty clear that there is exactly 1 assimilation period,
it may surprise you that the start is 1 second past 12Z. This is deliberate and reflects
the DART convention of starting intervals 1 second after the end of the previous interval.
The times in the netCDF variables reflect the defined start/stop of the period, regardless
of the time of the first/last observation.
Please note that none of the 'horizontal_wind' variables will have a rank histogram,
so they are not written to the netCDF file. ANY variable that does not have a rank
histogram with some observations will NOT have a rank histogram variable
in the netCDF file.
Now that you have the obs_diag_output.nc, you can explore it with
plot_profile.m, plot_bias_xxx_profile.m, or plot_rmse_xxx_profile.m,
and look at the rank histograms with
ncview or
plot_rank_histogram.m.
| Routine | Message | Comment |
|---|---|---|
| obs_diag | No first observation in sequence. | get_first_obs couldn't find a "first obs" in the obs_seq.final. |
| obs_diag | No last observation in sequence | get_last_obs couldn't find a "last obs" in the obs_seq.final |
| obs_diag | metadata incomplete | Couldn't find the index for the observation value in the observation sequence file. It is the only one that is required. |
| filter_get_obs_info | Vertical coordinate not recognized | It must be "surface", "pressure", or "height" |
| Convert2Time | namelist parameter out-of-bounds. Fix and try again | bin_width, bin_separation, and time_to_skip must have year = 0 and month = 0 |
none at this time
The RMSE of the vector wind velocity is being used.
The bias is actually the bias of the wind speed, with no regard to
direction. Seems like this is not consistent ...
Use log(p) instead of pressure for binning/plotting.
Hope to have a separate DART QC flag for observations outside the model state.
Right now, all observations that have a DARt forward operator fail (extrapolate, mainly)
get counted as observations that are rejected. Logically, these observations
are not possible because most (all?) DART observation operators cannot extrapolate.
If the specified levels are not in the proper range for the observations, no vertical profile is built.
The 'verbose' option should highlight the fact there are no observations in the specified vertical bins -
AND provide some sort of guidance about the min and max values of the vertical values.
N/A
DART software - Copyright UCAR. This open source software is provided by UCAR, "as is", without charge, subject to all terms of use at http://www.image.ucar.edu/DAReS/DART/DART_download


Learner Centered, Performance Based
Telling Ain’t Training
Telling Ain’t Training is a book that focuses on some effective techniques that can be used when teaching adult learners. It covers a wealth of topics, touching on cognitive psychology, the philosophy of knowledge, lesson planning, and effective assessment (among many others). The authors immediately stress their overarching philosophy of training from chapter one - learner centered, performance based. They argue that effective instruction must have these two qualities. Otherwise, students will not leave the lesson with the content that you intend for them to absorb.
On the surface, the idea of learner centered and performance based instruction seems pretty reasonable (perhaps even obvious). However, the devil is in the details. How do you build a learner centered lesson for a class of students? How do you effectively assess performance when everybody starts at different skill levels? Moreover, how can we do this in an online environment - one where you do not get any face-to-face interaction with the people you are tryinig to teach?
Many researchers and educators are hard at work trying to answer these questions. I would love to offer some ideas that I am going to try out on Cupcake Physics.
Learner Centered
Learner centered instruction means that the lesson at hand should be tailored to the knowledge and skills of the learner. This is not a new concept; educators have applied models like the Zone of Proximal Development to their lesson planning strategies for decades. However, I wanted to think about how I can implement learner-centered lessons to Cupcake Physics.
Keep the Lessons Small
It can be difficult to pay attention to a long lecture when you are sitting in a classroom. It can be near impossible to stay interested in the same lecture if it is recorded and played back on a computer. Unfortunately, there are too many distractions that can tear you away from a lesson online - even I find myself drifting away from online physics lectures when I am sitting at a computer, and I have spent a good part of my life focused on the subject! Thus, one of the ways that we keep the lesson learner-centered is to respect the fact that the online learning environment is fundamentally different than that of a classroom. Instruction must be focused and relevant in order to keep the learner interested when it is so easy to find another source online.
Do Less
This one seems a bit weird at first glance. In an era when we are being bombarded by a constant stream of information, it seems reasonable that we need to do more to keep up. However, learning takes place not just in the classroom, not just on Cupcake Physics, but during the quiet moments when we have a change to reflect on what we have already done. As I develop content on the website, I am going to try to find ways to slow down the pace of learning. The point of this is to encourage the learner to reflect on what has already been done. Self-reflection is one of the strategies that can be used to work towards a truly learner-centered curriculum.
Performance Based
Performance based instruction means that we want students to be able to apply the knowledge that they have learned from our training. The authors describe this as “being able to act and to achieve worthwhile, verifiable results” (Stolovitch 14). As always, easier said than done. A worthwhile result to a student (i.e. get an A on a test) might not match up directly with a worthwhile result from an instructor (i.e. develop knowledge about fundamental physics). A verifiable result to a student (i.e. get all of the questions on the homework correct) might not match up with the ideas of the instructor (i.e. parse a context-specific physics problem and come up with a best solution where there is no one correct answer). How might we measure performance on Cupcake Physics in order to maximize learning?
Constantly Collect User Input
Creating small chunks of instruction is only part of the story. Watching ten short videos back-to-back is not that different from watching one long one. There needs to be user input between each of those lessons to ensure that the lesson is still calibrated to the user’s skill level. This input can serve additional purposes. It might be a gateway to ensure that the student understands the preceding section before moving onto the next one. It might be a crossroad where the lesson goes in different directions, depending on the previous answer. Engagement is not a direct indicator of a learner-centered lesson, but it does imply that the lesson is within the skillset of the student.
Keep the Feedback Cycle Tight
Last but not least, the idea of tight feedback loops ties all of these ideas together. A tight feedback loop will (ideally) keep a student engaged in the lesson at hand because he or she will receive constant feedback on how much progress has been made. Additionally, it will give the program an opportunity to pivot and offer a new piece of content based on the answers given before. The classic idea of “questions at the end of the chapter” is not engaging in an online environment, nor does it does assess performance in an effective way. In a world where the answers to questions can be looked up on a whim, end of lesson questions no longer provide accurate feedback on the skillset of a student.
Closing Thoughts
There is a lot more to unpack in this book - far more than can be covered in a single blog post. As I develop content for the site over the next few weeks, I would love to continue exploring some of the lessons that I have taken away from this read. If you get a chance, please visit Cupcake Physics to try out the site. Until next time!
Works Cited
Stolovitch, Harold D. Telling Ain’t Training: Updated, Expanded, Enhanced. 2nd ed., Association for Talent Development, 2011


Every transaction on blockchain is public
Everyone can see your entire account history and balance
Most blockchains are very slow
Blockchains are inherently deterministic.
bytes32 blockHash = blockhash(block.number - 1);
uint32 winner = uint32(blockHash) % numOfTickets;
There is an inescapable time delay between the moment a blockchain transaction is broadcast to the network and the moment that that same transaction is mined into a block. During the intervening period, transactions sit in a pending transaction pool (mempool) waiting to be mined.
- Front-running, Griefing and the Perils of Virtual Settlement (Part 1) - Will Warren (0xProject)


Inside Nike's New S23NYC Studio, Where Sneaker Culture Is Amid a Digital Revolution
Nike VP Ron Faris talks to us about the future of the SNKRS app.
by Robert Marshall Jr.This week, Nike‘s S23NYC Studio — the in-house agency that oversees the brand’s SNKRS app — moved into a sprawling new space on South Street in New York City’s Seaport district, where it shares an impressive view of NYC’s East River with ESPN.
We were granted the opportunity to visit the digital studio’s HQ before it opened to its now 60+ employees, with the head of the outfit, Nike VP Ron Faris, offering a tour of the office.
SNKRS has grown exponentially since it launched back in 2015. In 2016, Nike purchased Faris’ Virgin Mega, which at the time was under Richard Branson‘s expansive Virgin umbrella, aiding the conglomerate with elevating fan communities and shopping experiences on mobile devices. The Swoosh was hoping Faris and team could do the same for its budding SNKRS application, as the brand was looking to not only reinvigorate the sneakerhead community through digital initiatives but also launch an aggressive direct-to-consumer strategy.
With this in mind, Faris’s goal was simple, although complex in theory. He wanted to bring the soul of sneaker culture to the digital world, by way of modernizing the excitement around “the hunt” or waiting in line at a boutique for a coveted sneaker. This has lead to in-app augmented reality features like Stash, where SNKRS members can visit a location or snap a photo of an object to unlock a pair of kicks.
“A lot of the kids that we talk to say how you cop the shoe is almost as important as the shoe itself; it’s about the experience of how you unlock it,” say Faris. “And AR is a really creative canvas to be able to unlock shoes in new and intriguing ways, whether it’s a billboard or other things. The canvas is the same but the art keeps changing.”
Now, with the success of SNKRS from a footwear and cultural standpoint, Faris and Nike are looking to bring this strategy to other areas of the Swoosh ecosystem, including apparel and real world experiences. “As we started getting bigger, we realized that we were unlocking a playbook for any fanatical community, not just the sneaker-obsessed. So as we broaden our mandate, we’re going to build a lot of these experiences for the running, training, skating and basketball obsessed.”
The playbook Faris is referring to is about tapping into fanatical communities with the 20/80 model – the top 20% are trendsetters who love to educate and inspire, while the other 80% are learning and discovering from them. SNKRS aims to bridge creatives with the communities they inspire not only through product, but through stories and experiences surrounding drops that allow them to shape the narrative entirely.
This is all achieved through the data its SNKRS members are willingly sharing with them. “As we started to scale the app, we got so much data that started to show us how we could tell stories and build product differently. Some of the product that we’ve done has been shaped by understanding the data of where our consumers reside.”
For instance, last year Nike analyzed the usage of its most-engaged SNKRS users in New York City and discovered that many lived in Dominican neighborhoods throughout the five boroughs. Nike then spoke with these communities and created a campaign celebrating their Dominican heritage, with the Air Force 1 “De Lo Mio” at the center. Faris and team then enlisted local Dominican photographers to document their Dominican cohorts wearing the shoe. When Nike finally released the sneaker, it quickly sold out.
“We’re starting to dig in deep with these communities and helping them shape what the story looks like. And as the community grows, we don’t want to be in the position where we tell them about our product, we want them to co-create the product and story.”
The latest frontier for SNKRS was announced just a few weeks ago: in partnership with 2K Sports, gamers will have the ability to unlock limited-edition sneakers — kicks you can wear, not your MyPLAYER — within the SNKRS app. The first of 10 gamer-exclusive releases throughout the coming season was a purple LeBron 17, which could only be unlocked after achieving a 98 rating for your MyPLAYER, surpassing LeBron’s rating of 97.
“It’s super exciting because it allows us to build tools where you can unlock hot sneakers in the app from any other community. It invites all of those communities in and learn more about sneakers.” Faris then tells us that he hopes to see this exclusive release model replicated for other realms of culture, like the food world.
All of this is then strengthened by the recent announcement of John Donahoe replacing Mark Parker as Nike’s CEO. Donahoe comes aboard as a digital mastermind, having most recently served as the president and CEO of ServiceNow, but has most famously held positions like chairman of PayPal, president and CEO of eBay, and CEO and worldwide managing director of Bain & Co.
“The combination of Mark and John is going to be fantastic for this next phase. Mark represents a lot of what got the brand to where it is now, through the art of storytelling with product, and John represents the future with his experience in digital,” says Faris.
So, what’s next for S23NYC and the new office space? “I’m excited for not only building new experiences and the data science, but a big part of this studio is bridging art and the science together,” says Faris. “We have a 1500 square-foot design zone, where we’re going to invite the community to come in and run clinics and workshops we want to bring to reality, which will then have a major affect on how we shape our stories, products and experiences.”
Faris even informs us of the latest feature coming to SNKRS: polling. “Where the community can give their take on a shoe we should bring back or what colorway they prefer on a model — local inspiration with global availability.”


EIPOPOAUNG
Hey! I'm EiPoPoAung.Coding had changed my world.It's not just about apps.Learning to code gave me problem-solving skillsand a way to communicate with others on a technical level.I can also develop websites and use my coding skills to get a better job.And I learned it all at Sololearnwhere they build your self-esteem and keep you motivated.Join me in this rewarding journey.You'll have fun,get help , and learn along the way!
"Declare variables,not war"
| Day | Mon | Tue | Wed | Thu | Fri |
|---|---|---|---|---|---|
| 8-8:30 | Learn | ||||
| 9-10 | Practice | ||||
| 1-1:30 | Play | ||||
| 3:45-5 | Code | ||||
| 6-6:15 | Discuss |


+------------------+
| 1 Aaaa Aaaa Aaaa |
| Aaa aa aaa aa |
| aaaaaa A aa aaa. |
| 2 |
| ####### c c C |
| ####### c c c |
| < ####### c c c |
| < ####### c c |
| < ####### . c |
| 3 ####### c |
+------------------+
Orientation Example:
====================
Above is a diagram of some (1) English and (2) Chinese text and a (3)
photo credit.
Upright Latin characters are represented as A and a. '<' represents a
latin character rotated anti-clockwise 90 degrees.
In the example above, both the English and Chinese paragraphs are
oriented so their "up" is the top of the page (page up). The photo credit
is read with one's head turned leftward ("up" is to page left).
The values of this enum match the convention of Tesseract's osdetect.h


vibe_elastic_logger.
logger
.ElasticInfo
public struct ElasticInfo
ElasticSearch server connection information.
Variables
hostName
string
Host name for the ElasticSearch server.
portNumber
ushort
Port for connecting to the ElasticSearch server.
typeName
string
The log message's "type" within an index.


👋 Hi, my name is Jae Ho YooN.
👨💻 I'm interested in Computer Vision in Autonomous-Driving.
💻 i'm putting down and share my studying entries on this Page and my Notion .
👥 I have much experience while doing many activities.
🤖 I develop autonomous-driving program that do not require driver control via OpenCV or pytorch.
📚 I try to study with a many thesis and lots of lectures such as CS231N or KOOC about things I learned.
📈 I am passionate about investing and I'm planning to share my experiences on a blog later on.


Select Foundry
(File)Click on File -> Select Foundry. The list of available processes appears. The initial design rule file is "default.tec". Various technologies are available from 1.2 down to 0.1 Ám. Click on the rule file name and the software reconfigures itself in order to adapt to the new process.
[Back]


| ZOOKEEPER-2008 |
System test fails due to missing leader election port |
Minor |
contrib-fatjar |
Kfir Lev-Ari |
Kfir Lev-Ari |
| ZOOKEEPER-2006 |
Standalone mode won't take client port from dynamic config |
Major |
server |
Hongchao Deng |
Hongchao Deng |
| ZOOKEEPER-2013 |
typos in zookeeperProgrammers |
Trivial |
documentation |
Tim Chambers |
Tim Chambers |
| ZOOKEEPER-2030 |
dynamicConfigFile should have an absolute path, not a relative path, to the dynamic configuration file |
Minor |
server |
Alexander Shraer |
Alexander Shraer |
| ZOOKEEPER-2026 |
Startup order in ServerCnxnFactory-ies is wrong |
Minor |
jmx, server |
Stevo Slavic |
Stevo Slavic |
| ZOOKEEPER-2054 |
test-patch.sh: don't set ulimit -n |
Major |
. |
Michi Mutsuzaki |
Michi Mutsuzaki |
| ZOOKEEPER-1917 |
Apache Zookeeper logs cleartext admin passwords |
Blocker |
. |
Flavio Junqueira |
Flavio Junqueira |
| ZOOKEEPER-2058 |
rat: exclude *.cer files |
Major |
build |
Michi Mutsuzaki |
Michi Mutsuzaki |
| ZOOKEEPER-2049 |
Yosemite build failure: htonll conflict |
Major |
. |
Till Toenshoff |
Till Toenshoff |
| ZOOKEEPER-2052 |
Unable to delete a node when the node has no children |
Major |
server |
Yip Ng |
Hongchao Deng |
| ZOOKEEPER-2060 |
Trace bug in NettyServerCnxnFactory |
Major |
server |
Ian Dimayuga |
Ian Dimayuga |
| ZOOKEEPER-2064 |
Prevent resource leak in various classes |
Critical |
. |
Ted Yu |
Ted Yu |
| ZOOKEEPER-2072 |
Netty Server Should Configure Child Channel Pipeline By Specifying ChannelPipelineFactory |
Major |
server |
Hongchao Deng |
Hongchao Deng |
| ZOOKEEPER-2111 |
Not isAlive states should be synchronized in ClientCnxn |
Major |
java client |
Hongchao Deng |
Hongchao Deng |
| ZOOKEEPER-1366 |
Zookeeper should be tolerant of clock adjustments |
Critical |
. |
Ted Dunning |
Hongchao Deng |
| ZOOKEEPER-1949 |
recipes jar not included in the distribution package |
Major |
recipes |
Rakesh R |
Rakesh R |
| ZOOKEEPER-2114 |
jute generated allocate_* functions are not externally visible |
Major |
c client |
Tim Crowder |
Tim Crowder |
| ZOOKEEPER-2073 |
Memory leak on zookeeper_close |
Critical |
c client |
Dave Gosselin |
Dave Gosselin |
| ZOOKEEPER-1952 |
Default log directory and file name can be changed |
Minor |
. |
nijel |
nijel |
| ZOOKEEPER-1893 |
automake: use serial-tests option |
Minor |
c client |
Michi Mutsuzaki |
Michi Mutsuzaki |
| ZOOKEEPER-2109 |
Typo in src/c/src/load_gen.c |
Trivial |
. |
Emmanuel Bourg |
Surendra Singh Lilhore |
| ZOOKEEPER-2146 |
BinaryInputArchive readString should check length before allocating memory |
Major |
. |
Hongchao Deng |
Hongchao Deng |
| ZOOKEEPER-2056 |
Zookeeper 3.4.x and 3.5.0-alpha is not OSGi compliant |
Major |
. |
Keren Dong |
Deiwin Sarjas |
| ZOOKEEPER-2157 |
Upgrade option should be removed from zkServer.sh usage |
Minor |
. |
J.Andreina |
J.Andreina |
| ZOOKEEPER-2029 |
Leader.LearnerCnxAcceptor should handle exceptions in run() |
Minor |
quorum |
Asad Saeed |
Rakesh R |
| ZOOKEEPER-2173 |
ZK startup failure should be handled with proper error message |
Major |
. |
J.Andreina |
J.Andreina |
| ZOOKEEPER-2174 |
JUnit4ZKTestRunner logs test failure for all exceptions even if the test method is annotated with an expected exception. |
Minor |
tests |
Chris Nauroth |
Chris Nauroth |
| ZOOKEEPER-2062 |
RemoveWatchesTest takes forever to run |
Major |
tests |
Flavio Junqueira |
Chris Nauroth |
| ZOOKEEPER-2171 |
avoid reverse lookups in QuorumCnxManager |
Major |
quorum |
Raul Gutierrez Segales |
Raul Gutierrez Segales |
| ZOOKEEPER-2182 |
Several test suites are not running during pre-commit, because their names do not end with "Test". |
Major |
tests |
Chris Nauroth |
Chris Nauroth |
| ZOOKEEPER-2190 |
In StandaloneDisabledTest, testReconfig() shouldn't take leaving servers as joining servers |
Major |
tests |
Hongchao Deng |
Hongchao Deng |
| ZOOKEEPER-1077 |
C client lib doesn't build on Solaris |
Critical |
build, c client |
Tadeusz Andrzej Kadłubowski |
Chris Nauroth |
| ZOOKEEPER-2156 |
If JAVA_HOME is not set zk startup and fetching status command execution result misleads user. |
Major |
scripts |
J.Andreina |
J.Andreina |
| ZOOKEEPER-2186 |
QuorumCnxManager#receiveConnection may crash with random input |
Major |
server |
Raul Gutierrez Segales |
Raul Gutierrez Segales |
| ZOOKEEPER-2124 |
Allow Zookeeper version string to have underscore '_' |
Major |
. |
Jerry He |
Chris Nauroth |
| ZOOKEEPER-2187 |
remove duplicated code between CreateRequest{,2} |
Minor |
c client, java client, server |
Raul Gutierrez Segales |
Raul Gutierrez Segales |
| ZOOKEEPER-2198 |
Set default test.junit.threads to 1. |
Minor |
build |
Chris Nauroth |
Chris Nauroth |
| ZOOKEEPER-2178 |
Native client fails compilation on Windows. |
Major |
c client |
Chris Nauroth |
Chris Nauroth |
| ZOOKEEPER-2197 |
non-ascii character in FinalRequestProcessor.java |
Minor |
. |
Michi Mutsuzaki |
Michi Mutsuzaki |
| ZOOKEEPER-2096 |
C client builds with incorrect error codes in VisualStudio 2010+ |
Major |
build, c client |
Vitaly Stakhovsky |
Vitaly Stakhovsky |
| ZOOKEEPER-2201 |
Network issues can cause cluster to hang due to near-deadlock |
Critical |
. |
Donny Nadolny |
Donny Nadolny |
| ZOOKEEPER-2213 |
Empty path in Set crashes server and prevents restart |
Blocker |
server |
Brian Brazil |
Hongchao Deng |
| ZOOKEEPER-706 |
large numbers of watches can cause session re-establishment to fail |
Critical |
c client, java client |
Patrick Hunt |
Chris Thunes |
| ZOOKEEPER-2212 |
distributed race condition related to QV version |
Critical |
quorum |
Akihiro Suda |
Akihiro Suda |
| ZOOKEEPER-2210 |
clock_gettime is not available in os x |
Major |
c client |
Michi Mutsuzaki |
Michi Mutsuzaki |
| ZOOKEEPER-2193 |
reconfig command completes even if parameter is wrong obviously |
Major |
leaderElection, server |
Yasuhito Fukuda |
Yasuhito Fukuda |
| ZOOKEEPER-2221 |
Zookeeper JettyAdminServer server should start on configured IP. |
Major |
server |
Surendra Singh Lilhore |
Surendra Singh Lilhore |
| ZOOKEEPER-2224 |
Four letter command hangs when network is slow |
Minor |
java client |
Arshad Mohammad |
Arshad Mohammad |
| ZOOKEEPER-1927 |
zkServer.sh fails to read dataDir (and others) from zoo.cfg on Solaris 10 (grep issue, manifests as FAILED TO WRITE PID). |
Major |
scripts |
Ed Schmed |
Chris Nauroth |
| ZOOKEEPER-2256 |
Zookeeper is not using specified JMX port in zkEnv.sh |
Minor |
scripts |
Arshad Mohammad |
Arshad Mohammad |
| ZOOKEEPER-2264 |
Wrong error message when secureClientPortAddress is configured but secureClientPort is not configured |
Minor |
server |
Arshad Mohammad |
Arshad Mohammad |
| ZOOKEEPER-2269 |
NullPointerException in RemotePeerBean |
Major |
jmx |
Arshad Mohammad |
Arshad Mohammad |
| ZOOKEEPER-2245 |
SimpleSysTest test cases fails |
Major |
. |
Arshad Mohammad |
Arshad Mohammad |
| ZOOKEEPER-2279 |
QuorumPeer loadDataBase() error message is incorrect |
Major |
quorum |
sunhaitao |
Arshad Mohammad |
| ZOOKEEPER-1803 |
Add description for pzxid in programmer's guide. |
Major |
documentation |
Leader Ni |
Arshad Mohammad |
| ZOOKEEPER-2244 |
On Windows zookeeper fails to restart |
Critical |
. |
Arshad Mohammad |
Arshad Mohammad |
| ZOOKEEPER-2239 |
JMX State from LocalPeerBean incorrect |
Major |
jmx |
Kevin Lee |
Kevin Lee |
| ZOOKEEPER-2302 |
Some test cases are not running because wrongly named |
Major |
tests |
Arshad Mohammad |
Arshad Mohammad |
| ZOOKEEPER-2142 |
JMX ObjectName is incorrect for observers |
Trivial |
. |
Karol Dudzinski |
Edward Ribeiro |
| ZOOKEEPER-1853 |
zkCli.sh can't issue a CREATE command containing spaces in the data |
Minor |
java client |
sekine coulibaly |
Ryan Lamore |
| ZOOKEEPER-2227 |
stmk four-letter word fails execution at server while reading trace mask argument. |
Major |
server |
Chris Nauroth |
Chris Nauroth |
| ZOOKEEPER-1371 |
Remove dependency on log4j in the source code. |
Major |
. |
Mahadev konar |
Arshad Mohammad |
| ZOOKEEPER-2211 |
PurgeTxnLog does not correctly purge when snapshots and logs are at different locations |
Major |
scripts |
Wesley Chow |
Arshad Mohammad |
| ZOOKEEPER-2311 |
assert in setup_random |
Major |
c client |
Marshall McMullen |
Marshall McMullen |
| ZOOKEEPER-2299 |
NullPointerException in LocalPeerBean for ClientAddress |
Major |
jmx |
Arshad Mohammad |
Arshad Mohammad |
| ZOOKEEPER-2295 |
TGT refresh time logic is wrong |
Major |
. |
Arshad Mohammad |
Arshad Mohammad |
| ZOOKEEPER-2281 |
ZK Server startup fails if there are spaces in the JAVA_HOME path |
Minor |
scripts |
neha |
neha |
| ZOOKEEPER-2340 |
JMX is disabled even if JMXDISABLE is false |
Minor |
. |
neha |
Arshad Mohammad |
| ZOOKEEPER-2229 |
Several four-letter words are undocumented. |
Major |
documentation |
Chris Nauroth |
Chris Nauroth |
| ZOOKEEPER-2252 |
Random test case failure in org.apache.zookeeper.test.StaticHostProviderTest |
Minor |
. |
Arshad Mohammad |
Timothy Ward |
| ZOOKEEPER-1029 |
C client bug in zookeeper_init (if bad hostname is given) |
Blocker |
c client |
Dheeraj Agrawal |
Flavio Junqueira |
| ZOOKEEPER-2243 |
Supported platforms is completely out of date |
Major |
. |
Ivan Kelly |
Chris Nauroth |
| ZOOKEEPER-2375 |
Prevent multiple initialization of login object in each ZooKeeperSaslClient instance |
Blocker |
java client |
yuemeng |
yuemeng |
| ZOOKEEPER-1991 |
zkServer.sh returns with a zero exit status when a ZooKeeper process is already running |
Minor |
scripts |
Reed Wanderman-Milne |
Biju Nair |
| ZOOKEEPER-2294 |
Ant target generate-clover-reports is broken |
Major |
build |
Charlie Helin |
Charlie Helin |
| ZOOKEEPER-2379 |
recent commit broke findbugs qabot check |
Blocker |
build |
Patrick Hunt |
Rakesh R |
| ZOOKEEPER-2133 |
zkperl: Segmentation fault if getting a node with null value |
Major |
contrib-bindings |
Botond Hejj |
Botond Hejj |
| ZOOKEEPER-2283 |
traceFile property is not used in the ZooKeeper, it should be removed from documentation |
Major |
documentation |
Arshad Mohammad |
Arshad Mohammad |
| ZOOKEEPER-2385 |
Zookeeper trunk build is failing on windows |
Blocker |
build |
Arshad Mohammad |
Arshad Mohammad |
| ZOOKEEPER-2388 |
Unit tests failing on Solaris |
Blocker |
tests |
Patrick Hunt |
Arshad Mohammad |
| ZOOKEEPER-2393 |
Revert run-time dependency on log4j and slf4j-log4j12 |
Blocker |
server |
Arshad Mohammad |
Arshad Mohammad |
| ZOOKEEPER-2195 |
fsync.warningthresholdms in zoo.cfg not working |
Trivial |
quorum |
David Fan |
Biju Nair |
| ZOOKEEPER-2364 |
"ant docs" fails on branch-3.5 due to missing releasenotes.xml. |
Blocker |
build, documentation |
Chris Nauroth |
Patrick Hunt |
| ZOOKEEPER-2413 |
ContainerManager doesn't close the Timer it creates when stop() is called |
Major |
server |
Jordan Zimmerman |
Jordan Zimmerman |
| ZOOKEEPER-2235 |
License update |
Blocker |
. |
Flavio Junqueira |
Flavio Junqueira |
| ZOOKEEPER-2330 |
ZooKeeper close API does not close Login thread. |
Major |
java client |
Arshad Mohammad |
Arshad Mohammad |
| ZOOKEEPER-2405 |
getTGT() in Login.java mishandles confidential information |
Blocker |
kerberos, security, server |
Patrick Hunt |
Michael Han |
| ZOOKEEPER-2450 |
Upgrade Netty version due to security vulnerability (CVE-2014-3488) |
Critical |
security, server |
Michael Han |
Michael Han |
| ZOOKEEPER-2297 |
NPE is thrown while creating "key manager" and "trust manager" |
Blocker |
server |
Anushri |
Arshad Mohammad |
| ZOOKEEPER-1460 |
IPv6 literal address not supported for quorum members |
Major |
quorum |
Chris Dolan |
Joseph Walton |
| ZOOKEEPER-2380 |
Deadlock between leader shutdown and forwarding ACK to the leader |
Blocker |
server |
Arshad Mohammad |
Arshad Mohammad |
| ZOOKEEPER-2366 |
Reconfiguration of client port causes a socket leak |
Blocker |
quorum |
Timothy Ward |
Flavio Junqueira |
| ZOOKEEPER-2458 |
Remove license file for servlet-api dependency |
Major |
. |
Flavio Junqueira |
Flavio Junqueira |
| ZOOKEEPER-2459 |
Update NOTICE file with Netty notice |
Blocker |
. |
Flavio Junqueira |
Flavio Junqueira |
| ZOOKEEPER-1898 |
ZooKeeper Java cli shell always returns "0" as exit code |
Critical |
java client |
Srinath Mantripragada |
Abraham Fine |
| ZOOKEEPER-2477 |
documentation should refer to Java cli shell and not C cli shell |
Major |
documentation |
Patrick Hunt |
Abraham Fine |
| ZOOKEEPER-1256 |
ClientPortBindTest is failing on Mac OS X |
Major |
tests |
Daniel Gómez Ferro |
Flavio Junqueira |
| ZOOKEEPER-2500 |
Fix compilation warnings for CliException classes |
Major |
. |
Abraham Fine |
Abraham Fine |
| ZOOKEEPER-2074 |
Incorrect exit codes for "./zkCli.sh cmd arg" |
Minor |
. |
Surendra Singh Lilhore |
Abraham Fine |
| ZOOKEEPER-2247 |
Zookeeper service becomes unavailable when leader fails to write transaction log |
Critical |
. |
Arshad Mohammad |
Rakesh R |
| ZOOKEEPER-2508 |
Many ZooKeeper tests are flaky because they proceed with zk operation without connecting to ZooKeeper server. |
Major |
tests |
Arshad Mohammad |
Arshad Mohammad |
| ZOOKEEPER-2548 |
zooInspector does not start on Windows |
Major |
contrib |
Arshad Mohammad |
Arshad Mohammad |
| ZOOKEEPER-2558 |
Potential memory leak in recordio.c |
Minor |
c client |
Michael Han |
Michael Han |
| ZOOKEEPER-2536 |
When provide path for "dataDir" with trailing space, it is taking correct path (by trucating space) for snapshot but creating temporary file with some junk folder name for zookeeper_server.pid |
Major |
server |
Rakesh Kumar Singh |
Rakesh Kumar Singh |
| ZOOKEEPER-2539 |
Throwing nullpointerException when run the command "config -c" when client port is mentioned as separate and not like new style |
Minor |
java client |
Rakesh Kumar Singh |
Rakesh Kumar Singh |


Building upon previously learned design fundamentals and information design concepts, this course aims to translate these skills into the interactive digital world through the use of the latest web technologies, focused around D3.
All of these lectures, along with all other class materials, are available at the following URL:
http://www.dave-landry.com/classes/artg5330-spring2018/
Utilizing the skills learned in the first half of the semester, you will each be creating a visualization platform using a dataset of your choosing.
Programming textbooks quickly become dated, and the best solutions are often found through collaboration and open dialogue.
I don't care which one you use.
...but I use Chrome because I think it's the best, and I'll make fun of you if you use Internet Explorer.
Terminal (Mac)
PowerShell (Windows)
macOS Terminal:
python --version
Windows PowerShell:
Get-Command python | fl *
“Git (/ɡɪt/) is a distributed revision control system with an emphasis on speed, data integrity, and support for distributed, non-linear workflows.”
(if you haven't already)
www.github.com
Your repository list is on the bottom right. You can think of a repository as a shared project folder.
Click the big green button.
You've created your first repository.
In a few minutes, it will automagically be published to http://<username>.github.io/
YOUR TASK: Change the contents of the README.md file
YOUR TASK: Create an index.html file and view it in a browser
Your repository has now been downloaded as a folder. Open the new folder in your text editor, where you should see a list of the files in a panel on the left.
Click on index.html to open it, change the text inside of it, and save it. Now let's try to upload that file back to Github.com.
Uploading files to your repository is called "pushing", and a push can contain several groups of changes called "commits".
Commits have comments attached to them that tells the world what was changed in those files.
Commit: "created a new logo"
index.html
logo.png
Commit: "changed background color and title"
styles.css
index.html
The program only lists files with changes. Check off the files you'd like to commit, and write a commit message.
Click "Commit to Master", and then "Push to Origin"
When a user types an address into a browser, that computer makes a request (client) to another computer that is running software that can send back files (server).
In order to proficiently write and test web code, our individual computer can act as both a server and a client.
So you've got updating files under your belt. But what if you want to view the changes in a browser BEFORE you push to the live site? These next few steps will get your computer set up as a local server to view these files.
PowerShell and Terminal are programs that interact with your computer through text commands. You can make folders, move files, and even launch applications.
Commands are issued by writing the command and pressing the return key. Only 1 command can be issued at a time.
When interacting with your computer through a shell, you are always issuing commands from a specific folder on your computer.
To move into a folder, use the change directory command:
cd /path/to/my/folder/
Use cd to go into your repository folder.
Python v2:
python -m SimpleHTTPServer
Python v3:
python -m http.server 8000
You should be able to see your site at the following URL:
http://localhost:8000/
Anytime you would like to stop the server from running (say, to commit some files to Git), you can cancel the process by pressing control + c on your keyboard while in the shell.
"HyperText Markup Language, commonly referred to as HTML, is the standard markup language used to create web pages."
Created by Tim Berners-Lee in 1995 as a way to distribute research papers.
"The language is written in the form of HTML elements consisting of tags enclosed in angle brackets (like <html>).
Browsers do not display the HTML tags and scripts, but use them to interpret the content of the page."
In the root directory of your project, create a new HTML file.
The Document Type Declaration is for HTML5 and "should" be the included as the first line of every HTML file.
<!DOCTYPE html>
If a declaration is not included, some browsers will revert to their legacy "quirks mode" for rendering.
The contents of every HTML file (except for the DOCTYPE) should be wrapped in <html> tags:
<!DOCTYPE html>
<html>
</html>
All webpages have a head and a body. The head contains information about the page (the brains) and the body contains information that will be visible to the user (the brawn).
<!DOCTYPE html>
<html>
<head>
</head>
<body>
</body>
</html>
Like the DOCTYPE before it, there is one more "required" tag that must be present in the head of every document.
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
</head>
<body>
</body>
</html>
Put something in the body of the page and make sure it's rendering to your browser.
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
</head>
<body>
Hello World
</body>
</html>
One of the many tags that the head supports is <title>.
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<title>My cool test page</title>
</head>
<body>
Hello World
</body>
</html>
Great for browsers, bookmarks, screen readers, and SEO
"Search engine optimization (SEO) is the process of affecting the visibility of a website or a web page in a search engine's unpaid results - often referred to as "natural," "organic," or "earned" results. In general, the earlier (or higher ranked on the search results page), and more frequently a site appears in the search results list, the more visitors it will receive from the search engine's users."
Search engines, in their spare time, go to every webpage they can find over the internet and either add or update their listings with the information it can read from the page.
That's known as "crawling".The more clear, organized, and descriptive the content is on a webpage, the more accurate the search engine's listing will become.
The basic structure of HTML was created based on what you would find in a research paper or Word document.
Most elements in HTML take up the full width of the screen (block), forcing the next element to be positioned below it on a new line.
Some elements, like <span>, are inline, which means that it will appear on the same line as the content before and after itself, never causing a line break.
HTML5 (the newest standard) introduced a new set of tags that are more literal than the dated and obscure tags like "div" and "span". These new tags include:
All SVG graphics need to be contained inside of an SVG element:
<body>
<svg></svg>
</body>
Shape properties are set as attributes of the element.
<body>
<svg>
<rect x="50px" y="100px" width="150px" height="100px" fill="red" />
</svg>
</body>
Next class will be January 29th.


import {ObserversModule} from '@angular/cdk/observers';
ContentObserver
An injectable service that allows watching elements for changes to their content.
| observe | |
|---|---|
|
Observe content changes on an element. |
|
| Parameters | |
|
element Element
|
The element to observe for content changes. |
| Returns | |
Observable<MutationRecord[]>
|
|
| observe | |
|---|---|
|
Observe content changes on an element. |
|
| Parameters | |
|
element ElementRef<Element>
|
The element to observe for content changes. |
| Returns | |
Observable<MutationRecord[]>
|
|
CdkObserveContent
Directive that triggers a callback whenever the content of its associated element has changed.
Selector: [cdkObserveContent]
Exported as: cdkObserveContent| Name | Description |
|---|---|
@Input()
|
Debounce interval for emitting the changes. |
@Input('cdkObserveContentDisabled')
|
Whether observing content is disabled. This option can be used to disconnect the underlying MutationObserver until it is needed. |
@Output('cdkObserveContent')
|
Event emitted for each change in the element's content. |


Lag Today
Posted: Fri Dec 11, 2009 7:40 pm
About 11:30 this morning there was a spike in lag which Karl said was the server backing up. Ever since then I have been experiencing a slowly increasing amount of lag, with increasing frequency to kicking me off the server.  I was hunting in hatchlings when this was happening, but it was occuring on my newb in trolls as well. Dying and standing up have been especially laggy.


{% comment %} Generic pagination controls to show at the top or bottom of a page. Context: page_obj -- default for Django Paginator object. num_links -- [Optional] number of visible page links before and after the active link (defaults to 3) {% endcomment %} {% comment %} Fucking complicated, is what this mess of `with` blocks is. In the end, this converts `num_links` (or the default `3` integer) into other ints including the buffer (`num_links-1`) and negative numbers for both `num_links` and that buffer. These kind of conversions are not easy to do within templates, and would be better served as context variables processed within the view. Nonetheless, doing it here allows for template-level control over number of pages to display. If nothing else, all you need to do to adjust number of visible links is change the default in the first `with` block below. {% endcomment %} {% with n_links=num_links|default:3 %} {% with n_links_buff=n_links|add:"-1" %} {% with n_links_slug=n_links|slugify n_links_buff_slug=n_links_buff|slugify %} {% with n_links_neg="-"|add:n_links_slug n_links_buff_neg="-"|add:n_links_buff_slug %}


--- layout: post status: publish published: true title: D820, Ibex and Hibernate author: display_name: Tomas Restrepo login: tomasr email<EMAIL_ADDRESS>url: http://winterdom.com/ author_login: tomasr author_email<EMAIL_ADDRESS>author_url: http://winterdom.com/ wordpress_id: 1039 wordpress_url: http://winterdom.com/2008/12/d820ibexandhibernate date: '2008-12-29 18:35:50 +0000' date_gmt: '2008-12-29 18:35:50 +0000' categories: - Linux tags: [] comments: [] ---
I recently mentioned that I had installed Ubuntu 8.10 Intrepid Ibex on my Dell D820 and that it was working mostly fine, except for the hibernate functionality.
Last Monday (Dec. 22nd), the mainboard on my D820 died. Fortunately, it was still under warranty, so I called Dell Service the next morning and on the afternoon of the 24th someone came by and replaced the board. Definitely a lot speedier than I expected and it was a very nice touch.
Anyway, couple of days later I figured out why hibernate didn’t work: I had no swap partition at all! I had simply ignored it when installing Ubuntu because I wasn’t sure if I was going to keep it that way or format it later to install something else.
So I fired up GParted, added a new swap partition and enabled it, roughly following this guide, properly adjusted to my system.
After that, I just needed to edit my GRUB configuration (/boot/grub/menu.lst
) so that the kernel boot options included the “resume” option pointing to the new swap partition (in my case, “resume=/dev/sda4
”).
For the most part, seems to be working nicely now. I still wish the system had just issued a warning or error when I tried to hibernate it without an active swap partition. I mean, if it’s not supposed to work, it shouldn’t fail so miserably.


- getAllUsers() - Static method in class org.commonvox.insight.anomaly_detector.User
-
Returns a Collection of all instantiated User objects in user-id order.
- getAnomalyData(Integer) - Method in class org.commonvox.insight.anomaly_detector.PurchaseManager
-
If submitted purchase amount is an anomaly, returns a two-element array consisting of (a) mean
and (b) standard deviation that formed the basis for the anomaly computation; if submitted
purchase amount is NOT an anomaly, returns null.
- getAnomalyData(Integer) - Method in class org.commonvox.insight.anomaly_detector.User
-
Determines whether the submitted purchase amount is an anomaly, and if so, returns a
two-element array consisting of (a) the mean, and (b) the standard deviation that formed the
basis for the anomaly computation; if submitted purchase amount is NOT an anomaly, returns null.
- getDegreesOfSeparation() - Static method in class org.commonvox.insight.anomaly_detector.User
-
Get the class-variable setting for "degrees of separation", which establishes how User networks
are derived.
- getFriends() - Method in class org.commonvox.insight.anomaly_detector.User
-
Returns NavigableSet of user's friends
- getId() - Method in class org.commonvox.insight.anomaly_detector.User
-
Returns id, which uniquely identifies User instance.
- getNetwork() - Method in class org.commonvox.insight.anomaly_detector.User
-
Returns NavigableSet of all users in this user's network.
- getOrCreateUser(String) - Static method in class org.commonvox.insight.anomaly_detector.User
-
Either gets existing User identified by the submitted id, or if no such User exists, creates
and returns a new User instantiated with the submitted id.
- getPurchaseMap() - Method in class org.commonvox.insight.anomaly_detector.PurchaseManager
-
Returns the internally-managed purchaseMap, containing recent purchases up to the
capacity established by
the purchase threshold
.
- getThreshold() - Static method in class org.commonvox.insight.anomaly_detector.PurchaseManager
-
Value of threshold class variable, representing the threshold (max count) of purchases to be
utilized in
anomaly assessment
.


First Project
Quinta da Regaleira is a quinta located near the historic centre of Sintra, Portugal. It is classified as a World Heritage Site by UNESCO within the "Cultural Landscape of Sintra". Along with the other palaces in the area such as the Quinta do Relógio, Pena, Monserrate and Seteais palaces, it is considered one of the principal tourist attractions of Sintra.
Second Project
A volcano is a rupture in the crust of a planetary-mass object, such as Earth, that allows hot lava, volcanic ash, and gases to escape from a magma chamber below the surface.
Third Project
Cowboy is a 2 years old Pit mix from Chicago


If you want to customize Core_CSS or add new widget or css class then you can add your own js or less files and run Grunt.
Before use Grunt you must install nodejs and last npm.
cmd:\> cd path_to\Core_CSSInstall grunt:
cmd:\> npm install -g grunt-cliInstall project dependencies:
cmd:\> npm installAdd your files:
cmd:\> grunt
To change grunt tasks you can modify Grunfile.js in project directory.


{% extends 'canon/base.html' %} {% from "pagination.html" import pagination %} {% block content %}
This is a list of common questions within the Python tag. Each question includes some canonical SO questions that can be used to close-vote any new questions that match. If you have any suggestions then please come see us in chat.
{{ item.excerpt }}


WITNESS LIST Criminal Justice Committee April 22, 1997-9:00A SB 279 For: Bollom, James (self), Houston True, Jeff (TX Quarter Horse Ass'n), Austin On: Scott, Wayne (TDCJ), Huntsville Registering, but not testifying: For: Powers, Bill (Texas Farm Bureau), Austin Small, Ed (TX & SW Cattle Raisers), Austin SB 548 For: Reddoch III, E.B. (TX State Rifle Assoc.), Richardson Registering, but not testifying: On: Wallace, Sherri (Dallas DA's Office), Dallas SB 612 For: Willstrop. Geff W. (self), Universal City SB 919 Registering, but not testifying: On: Kelly, Laurel (Office of the AG), Austin Lyons, Laura (TAASA), Austin McKenzie, Cecelia (Sex Assault Prev Prog.), Austin SB 1011 Registering, but not testifying: On: Boatright, David M. (TX DPS), Austin Hatcher, Johnny R. (TX DPS), Austin SB 1120 Registering, but not testifying: On: Boatright, David M. (TX DPS), Austin Brubaker, James W. (TX DPS), Austin Hatcher, Johnny R. (TX DPS), Austin SB 1728 For: McCormick, Michael J. (Court of Crim. Appeals), Austin Registering, but not testifying: For: Wetzel, Richard (Court of Crim. Appeals), Austin SB 1886 Registering, but not testifying: For: Gutierrez, Hector (City of El Paso), El Paso On: Kyser, Ben (TX DPS), Austin


Langrisser I & II Lands A 2020 Release Date In The West
Check out the changes
by Damien McFerranIt has been confirmed that Langrisser I & II will launch on March 10th, 2020 in North America and March 13th in Europe, while Australia and New Zealand will get the game on March 20th.
A double-pack which remasters two classic Mega Drive SPRGs from the '90s, this collection features new visuals and voice acting, as well as the chance to toggle between 'Classic' and 'Remastered' art styles, as demonstrated in the trailer above.
Originally developed by Masaya Games, the Langrisser series began life in 1991, and would make its way to North America under the title Warsong. Subsequent entries – which would arrive on the Super Famicom, PC-FX, Saturn and PlayStation – have remained exclusive to Japan. However, Langrisser Re:Incarnation Tensei changed all of that in 2015 when it was localised for western release (sadly, it was largely awful).


Admin Page
Wrong Transaction is detected.
DEBT is not setted yet.
ROE is not setted yet.
Target DEBT:
10
20
30
40
50
60
70
80
90
100
Target ROE:
1
2
3
4
5
6
7
8
9
10
Real DEBT is:
Warning: Real DBET is Great than target Value
Real DBET is less than or equals as target Value
Real Roe is:
Warning: Real ROE is less than target Value
Real ROE is greater than or equals as target Value
Chain Name
Index
Content
hash
Updated Time
Comment
View History
Capital Chain
View History
Reputation Chain
View History
Value Chain
View History
Bank Chain
View History
Rank Chain
View History
This ABS belong to CDO:
View Details
This ABS belong to CDOs:
View Details


The dataset is obtained from the public data from data.gov website under the domain consumer-complaint-database
. The Consumer Complaint Database is a collection of complaints about consumer financial products and services that we sent to companies for response. The database generally updates daily. So, each day when we download the dataset, it may be larger than the previous dataset.
The dataset has above 1 million rows and 18 columns out of which, for the text data category classification, we are only interested in two features: Product
and Consumer complaint narrative
.
This project aims to accurately classify the Product category of the complaint. There are more than 10 categories of the product such as Mortgage
, Debt collection
and so on. Our aim is to read the text complaint and classify as on of these category.
NOTE
Originally there are more than 10 categories in original database, but some of the categories are
ambiguous, such as there are three different categories Credit card
, Prepaid card
, and Credit card or prepaid card
. If we have a given complaint about credit card, what should it be classified as? Credit card
or Credit card and prepaid card
? To avoid this problem the ambiguous categories are merged into one single categories and finally we have only 10 different categories.
For a sample of 2,000 data, the category distribution looks like this:
Usually the written text is full of informal language and requires cleaning the text before we proceed with analyzing the text. For example, we need to remove the STOPWORDS and expand the contractions. Data cleaning strategy:
areYou
==> are You
You
==> you
you're
==> you are
hi !
==> hi
gr8
==> gr
ha ha
==> ha
I am good
==> good
apples
==> apple
For the text processing tasks (NLP), we usually use a method called Term Frequency - Inverse Document Frequency
.
Term Frequency:
This gives how often a given word appears within a document.
TF = Number of times the term appears in the doc
----------------------------------------------
Total number of words in the doc
Inverse Document Frequency:
This gives how often the word appears across the documents.
If a term is very common among documents (e.g.,the
, a
, is
),
then we have low IDF score.
Number of docs the term appears
DF = -----------------------------------
Total number docs in the corpus
But, conventionally, document frequency (Df) is defined as log of ratio,
Number of docs the term appears
DF = ln ( ----------------------------------)
Total number docs in the corpus
Term Frequency – Inverse Document Frequency TF-IDF:
TF-IDF is the product of the TF and IDF scores of the term.
TF
TF-IDF = ------
DF
We can use scikitlearn text vectorizer class sklearn.feature_extraction.text.TfidfVectorizer
to get
the vectorized form of given text data. Then using feature selection (sklearn.feature_selection.chi2
) we get
following top most unigrams and bigrams for each categories:
We can not use the raw text data as the input for scikit-learn
classifiers.
We first need to vectorize them and convert the words to number. Here, in this
project I have used the Tf-idf vectorizer with ngram of (1,2) and tried various
classifiers. Among many classifiers, I found svm.LinearSVC gave me the best accuracy.
For the 2019 data with sampling of 2000 samples with random seed of 100, I got the
accuracy of 0.8125 for the test data. For the full data of 2019 (124,907 almost 125k)
after splitting train-test as 80%-20%, I got the accuracy of 0.8068.
Here, we have so far used only the small portion of the data (2,000 samples out of million samples) and used scikit-learn
models for the text analysis. But, for the real world data, we may need to use all the data for better performances.
For large data, pandas crashes and we need to look for alternative methods such as Amazon AWS or IBM Watson. Also, we can use the open source modules such as dask
or pyspark
which can scale up to multiple gigabytes of data. For this project, I have used both pyspark
and Amazon AWS servers.
NOTE:
Pyspark
is an immature library. It was borrowed from scala and many functionalities are still need to be implemented. For example, while reading the complaints.csv
file, using pandas we can simply use pd.read_csv
, however, pyspark is not sophisticated enough to read the csv file automatically when it has multiline. To circumvent these obstacles we can use spark read option with multiLine=True, escape='"'
.
For text data processing using pyspark
, here I have used following pipelines:
from pyspark.ml.feature import Tokenizer,StopWordsRemover,HashingTF,IDF
tokenizer = Tokenizer().setInputCol("complaint").setOutputCol("words")
remover= StopWordsRemover().setInputCol("words").setOutputCol("filtered").setCaseSensitive(False)
hashingTF = HashingTF().setNumFeatures(1000).setInputCol("filtered").setOutputCol("rawFeatures")
idf = IDF().setInputCol("rawFeatures").setOutputCol("features").setMinDocFreq(0)


2020/03/16
First “smart” lockdown, we all moved to remote working from one day to another but it was still fun and games.
2020/03/19
Addition of the office monitor, the migration starts. The Kinect comes out of retirement to finally be useful step-up video calls.
2020/04/10
2020/05/09
2020/07/08
2020/11/14
The only advantage of a glass desk is to take the red garland off the street window, to write dark poetry in a new first-class ergonomic chair.
2021/01/20
2021/01/29
Serious gear up!
Bye-bye the worthless glass desk, the brand-new desk can stand!
The all-black setup is complete, even the MacBook followed the dark side. Spoiler: it only lasted an evening before colours came ruining it.
2021/01/30
Multicolour LED light strip, backlight mechanical keyboard and Rubik’s cube are required for a geek ensemble.
2021/03/14
New desk mat, new monitor stand, Kinect on top and new massage ball; is it the ultimate setup?
Don’t mention cable management, it’s a passionate journey that should not come to an end.
Take care in home, stay safe out there.


Welcome to the homepage. Learn more about my exercise page.
I play the sample recording on this page.


This page allows you to configure common global Jabber server options.
- Your server's hostname
The hostname that clients will use to connect to your Jabber server. This
should be the fully qualified domain name, like jabber.foo.com.
- Error log file and Error log fomat
These control which file (relative to the server base directory) is used
for logging errors, and the format that the log is in.
- Access log file and Access log fomat
These control which file (relative to the server base directory) is used
for recording client logins and logouts, and the format that the file
is in.
- Process ID file
The file to which the server will write its process ID.


LF Email sending class provides an easy way to compose emails with multiple attachments and multiple recipients.
$this->load->systemLibrary('Email');will load the class and the email object will be available within your code.
Within your controller:
$this->load->systemLibrary('Email'); $this->Email->from('me@myserver.com'); $this->Email->to('recipient@remoteserver.com'); $this->Email->subject('Test mail'); $this->Email->message('This is my test message'); $this->Email->send();will send an email to<EMAIL_ADDRESS>from<EMAIL_ADDRESS>with the subject Test mail and with the content This is my test message.
$this->Email->from('me@mymail.com');
$this->Email->replyTo('you@yourmail.com');
$this->Email->to('you@yourmail.com');
$list =<EMAIL_ADDRESS><EMAIL_ADDRESS>, 'three@yourmail.com'); $this->Email->to($list);
$this->Email->subject('This is my subject');
$this->Email->message('This is my message');
$this->Email->send();sends your email.
$this->Email->attach('/var/www/test/test.jpg'); $this->Email->attach('/var/www/test/test.odt'); $this->Email->attach('/var/www/test/test.pdf');


| Uniplex 171 General Information
-
| Synonyms: |
p-Toluenesulfonamide; o-toluenesulfonamide; 4-Methylbenzene sulfonamide, 2-methylbenzene sulfonamide; 4-methylbenzenesulfonamide; 2-methylbenzenesulfonamide |
| Molecular Formula: |
C7H9NO2S |
| CAS Registry No.: |
70-55-3; 88-19-3 |
| Chemical Category: |
Sulfonamides |
Usage and Performance Characteristics
-
| Manufacturer: |
Unitex Chemical Corporation |
| Outstanding Properties: |
Improved PA flexibility and PU hardness |
| Used for Polymeric Materials: |
Melamine, urea, phenolic, polyamide |
| Misc. Info on Composition: |
70% p-Toluenesulfonamide CAS# 70-55-3; 30% o-toluenesulfonamide CAS# 88-19-3 |
Physical Properties
-
| Aggregate State at 25°C: |
Solid particles |
| Dielectric Constant at 1 MHz: |
3.51 |
Chemical Properties
-
| Hildebrand Solubility Parameter, MPa1/2: |
33.17 |
Optical Properties
Hazard-Related Data
-
| OSHA Hazard Class: |
Not hazardous 29CFR 1910.1200 |
| Flash Point Temperature, °C: |
216 |
| HMIS Personal protection: |
Safety glasses, gloves, lab. coat |
Environmental Parameters
-
| Theoretical Oxygen Demand, ThOD, g/g: |
2.06 |
Plasticizers Database (2nd Electronic Edition) Copyright © 2004 by ChemTec Publishing |


Barebones Presentation
This example contains the bare minimum includes and markup required to run a reveal.js presentation.
No Theme
There's no theme included, so it will fall back on browser defaults.
## Markdown support
Write content using inline or external Markdown.
Instructions and more info available in the [readme](https://github.com/hakimel/reveal.js#markdown).


- 1
- F. Porcelli, R. Stankiewicz, W. Kerner, and H. L.
Berk.
Solution of the drift-kinetic equation for global plasma
modes and finite particle orbit widths.
Physics of
Plasmas, 1(3):470-480, 1994.
- 2
- Allen H. Boozer.
Physics of magnetically
confined plasmas.
Rev. Mod. Phys., 76:1071-1141, Jan
2005.
- 3
- Philipp Lauber.
Linear
gyrokinetic descritpion of fast particle effects on the MHD stability in
Tokamaks.
PhD thesis, Max-Planck institute for plasma physics,
2003.
- 4
- Y. Todo.
Properties of energetic-particle
continuum modes destabilized by energetic ions with beam-like velocity
distributions.
Physics of Plasmas (1994-present),
13(8):-, 2006.
YouJun Hu
2014-05-19


Spectrum Educational Tool
an online resource training for student teachers
{% if username == "guest" %}
Welcome Guest
LOGIN
{% else %}
Welcome {{username}}
LOGOUT
{% endif %}
Home
>Life Cases
Threatening Student
Negative Group Dynamics
Bullying Student
Disrespectful Student
Attention Seeking Student
Emotionally Neglected Student
Home
Case Studies
Life Cases


--- layout: post title: "No Title" ---
Would I have this much self-esteem if I weren’t so awesome? That’s for God to answer tonight after I beat him at racquetball.


Gandhi learnt scavenging in South Africa. His friends there lovingly called him thegreat scavenger. Mahatma Gandhi said, “Everyone must be his own scavenger.” (The mind of Mahatma Gandhi) Aftera three-year stay in South Africa, he returned to India to fetch his wife andsons. Plague had broken out in the Bombay Presidency at that time. There was achance of it spreading to Rajkot. Gandhi immediately offered his service for improving the sanitation of Rajkot. He inspected every home and stressed theneed of keeping the latrines clean. The dark, filthy, stinking pits infested with vermin horrified him. In some houses belonging to the upper class, gutterswere used as a privy and the stench was unbearable. The residents were apathetic. Poor untouchables lived in cleaner homes and responded to Gandhi'spleadings. Gandhi suggested the use of two separate buckets for urine andnight-soil and that improved the sanitation.
After twenty years stay inthat alien land, Gandhi, at 46, finally returned to India with his group.During his visit to Kumbh Mela at Hardwar that year, he with his Phoenix boysserved as bhangis at the mela. The same year Gandhi visitedthe Servants of India Society's quarters at Poona. The members of the smallcolony saw him cleaning the latrines one morning. They did not like it. ButGandhi believed that work of this kind qualified one for Swaraj. More than once he toured allover India. Wherever and whenever he went, he found unsanitary conditions insome form or other and said that though few could afford shoes, it wasunthinkable to walk barefoot in India. Even in a city like Bombay, peoplewalked about the streets under the fear of being spat upon by the occupants of buildingsaround. The filth and stench of public urinal and latrines on railway stations andin dharmashalas were awful. Gandhi deplored the passengers' habit ofdirtying the railway compartments. The roads used by the poorvillagers and their bullocks were always ill-kept. He saw people taking a dipin a sacred pond without caring to know how dirty that bathing place or thewater was. They themselves dirtied the river-banks. It hurt him to see the marblefloor of Kashi Vishwanath Temple set with silver coins collecting dirt. In talking withmunicipalities, Gandhi often said; “I congratulate you on your spacious roads,your splendid lighting and your beautiful parks. But a municipality does not deserve to exist which does not possess model water-closets and where streetsand lanes are not kept clean all the hours of the day and night. Have you everthought of the conditions in which the sweepers live?"
Mahatma Gandhi always dreamt of a clean India , according to him keeping the society clean was much more important than independence.. And in order to fulfill this unfulfilled dream of the father of our nation , we thought of creating something that would change the mindset of the people, that would make them realize about their misplaced social values so that they should take initiative for the betterment of the society. Society should be reformed , the culprits should be taught the social values , Because what we are, truly reflects in the society. As In order to see the change we have to be the change, thus was House Of Change initiated.