Note
This tutorial was generated from a Jupyter notebook that can be downloaded here.
The null space¶
A key principle in the problem of mapping the surfaces of stars and planets is the idea of a null space. The null space of a (linear) transformation is the set of input vectors that result in a zero vector as output. In the context of mapping, the null space comprises the spherical harmonics (or combinations of spherical harmonics) that do not affect the observed flux whatsoever.
A trivial example is the \(Y_{1,-1}\) spherical harmonic, which does not project into the light curve for rotations about the vertical (\(\hat{y}\)) axis:
It is clear that as this object rotates, the total flux does not change (and in fact is exactly zero). That’s because the \(Y_{1,-1}\) harmonic is perfectly symmetric under such rotations.
It might be hard to think of other harmonics that behave this way, but in fact the vast majority of spherical harmonics are usually in the null space of the light curve problem. Let’s take a deeper look at this. First, we’ll load starry
and generate a high resolution map of the Earth.
[5]:
import matplotlib.pyplot as plt
import numpy as np
import starry
starry.config.lazy = False
starry.config.quiet = True
[6]:
map1 = starry.Map(30)
map1.load("earth", sigma=0.05)
map1.show(projection="rect")
Let’s now compute the light curve of this map as we rotate it one full cycle about the \(\hat{y}\) direction:
[7]:
theta = np.linspace(0, 360, 1000)
flux1 = map1.flux(theta=theta)
plt.figure(figsize=(12, 5))
plt.plot(theta, flux1)
plt.xlabel(r"$\theta$ [degrees]", fontsize=24)
plt.ylabel("Flux [arbitrary units]", fontsize=24);
At phase 0, the prime meridian is facing the observer, and the flux drops as the Atlantic and then Pacific oceans come into view. The flux then peaks when Asia is in view, and that’s the light curve of the Earth.
Now, let’s create a new, identical map of the Earth:
[8]:
map2 = starry.Map(30)
map2.load("earth", sigma=0.05)
But this time, we’ll change all coefficients corresponding to odd degrees above 2…
[9]:
for l in range(3, map2.ydeg + 1, 2):
map2[l, :] = 1e-3 * np.random.randn(len(map2[l, :]))
… as well as all coefficients corresponding to negative values of m
:
[10]:
for l in range(1, map2.ydeg + 1):
map2[l, -l:0] = 1e-3 * np.random.randn(len(map2[l, -l:0]))
Here’s what this silly map looks like:
[11]:
map2.show(projection="rect")
It doesn’t really look anything like the Earth (though a sharp eye might spot the outline of Africa and some other familiar features – barely). If you’re wondering why we did this, let’s plot the light curve of this new map next to the light curve of the original map of the Earth:
[12]:
flux2 = map2.flux(theta=theta)
plt.figure(figsize=(12, 5))
plt.plot(theta, flux1, label="map1")
plt.plot(theta, flux2, "--", label="map2")
plt.legend()
plt.xlabel(r"$\theta$ [degrees]", fontsize=24)
plt.ylabel("Flux [arbitrary units]", fontsize=24);
Their light curves are identical. Even though the maps look completely different, the actual differences between the two maps lie entirely in the null space (by construction). This means we have no way of distinguishing between these two maps if all we have access to is the light curve.
To drive this point home, here’s what the two maps look like side-by-side:
[13]:
map1.show(theta=np.linspace(0, 360, 50))
map2.show(theta=np.linspace(0, 360, 50))
The total amount of flux at any given time – equal to the brightness integrated over the entire disk – is the same in both cases, even though the surfaces look nothing alike.
For objects rotating along an axis perpendicular to the line of sight (as in the example above), the null space consists of all of the \(m < 0\) harmonics, as well as all of the harmonics of degree \(l = 3, 5, 7 ...\):
The null space for this problem is indicated by the translucent harmonics above. (Note that in general certain linear combinations of the remaining harmonics are also in the null space, so the problem is actually worse than in the example above).
The size of the null space¶
Because the light curve model in starry
is linear, it’s actually easy to compute the size of the null space for a given problem. Consider how the flux \(\mathbf{f}\) is computed in starry
:
where \(\mathbf{A}\) is the design matrix (which you can obtain from map.design_matrix()
) and \(\mathbf{x}\) is just the (amplitude-weighted) spherical harmonic coefficient vector. When solving this system for \(\mathbf{x}\), the matrix we end up needing to invert looks like
where \(\Sigma\) is the data covariance matrix. The object \(\mathbf{I}\) is the Fisher information matrix and quantifies the maximum amount of information we can learn given our observations \(\mathbf{f}\). While there’s a lot we can do with \(\mathbf{I}\), the most important thing in our case is the size of the null space, which is equal to the number of rows (or columns) minus the rank of this matrix.
There’s lots of ways to think about why this is the case. Note that \(\mathbf{I}\) is invertible if (and only if) it is full rank, meaning the null space is zero. That’s the condition for a well-posed problem, in which we can learn everything there is to know about \(\mathbf{x}\) from our observations \(\mathbf{f}\). In nearly all cases in the mapping problem, \(\mathbf{I}\) is not full rank, meaning it has linearly dependent rows or columns. For each such column, that’s one fewer term (or combination of terms) we are able to constrain in \(\mathbf{x}\). In other words, the size of the null space is equal to the number of linearly dependent rows/columns in \(\mathbf{I}\).
Note, importantly, that in the limit that our measurements are uncorrelated, the rank of \(\mathbf{I}\) is equal to the rank of \(\mathbf{A}^\top \mathbf{A}\) (i.e., independent of the uncertainties). Note also that this quantity does not depend on either \(\mathbf{f}\) or \(\mathbf{x}\) – just the design matrix!
Let’s take a look at the size of the null space in the problem above. We’ll use numpy.linalg.matrix_rank
to compute the matrix rank.
[15]:
def compute_info(A):
"""Compute some information about the null space of the design matrix A."""
# Get the Fisher information & compute its rank
I = A.T.dot(A)
R = np.linalg.matrix_rank(I)
# Number of coefficientss
C = I.shape[0]
# Size of null space
N = C - R
# Fractional size of null space
F = N / C
# Show it graphically
fig, ax = plt.subplots(figsize=(6, 0.3))
ax.set_xlim(0, 1)
ax.axis("off")
ax.axvspan(0, 1 - F, color="C0")
ax.axvspan(1 - F, 1, color="red")
ax.annotate(
"{}/{}".format(R, C),
color="C0",
fontsize=10,
xy=(-0.025, 0.5),
xycoords="axes fraction",
va="center",
ha="right",
)
ax.annotate(
"{:.0f}%".format(100 * F),
color="w",
fontsize=10,
xy=(1 - 0.5 * F, 0.5),
xycoords="axes fraction",
va="center",
ha="right",
)
[16]:
A = map1.design_matrix(theta=theta)
compute_info(A)
The red bit represents the null space, while the blue bit represents the terms we can constrain from the data (the “image” of the linear transformation). Our problem has 961 free coefficients, but the rank of the Fisher matrix is only 33. That means that 928 of the coefficients (or, more precisely, 928 linear combinations of coefficients) lie in the null space. In other words, 97 percent of the information about the map cannot be probed from the light curve in this case.
Yep. 97 percent.
So, how can we possibly map other stars and planets?¶
There are several ways to get around the information issue, and they all involve finding ways to break degeneracies. Below is a non-exhaustive list of ideas.
Lower degree maps¶
The size of the null space increases pretty steeply with the degree of the map. That’s because we can typically only constrain up to two terms per spherical harmonic degree from a light curve (a cosine term and a sine term); but the number of terms increases quadratically with the degree of the map. Let’s look at the size of the null space for a 5th degree map (instead of 30th):
[17]:
map = starry.Map(5)
A = map.design_matrix(theta=theta)
compute_info(A)
It’s better – now the size of the null space is only 81%. The high order terms are usually the hardest ones to constrain!
Inclined maps¶
There’s a perfect degeneracy for maps seen at an inclination of 90 degrees, since there’s no way to differentiate between the northern and southern hemispheres of the map – they contribute equally to the light curve. As long as there’s a little inclination, that degeneracy is broken. Consider a map seen at 60 degrees inclination:
[18]:
map = starry.Map(5)
map.inc = 60
A = map.design_matrix(theta=theta)
compute_info(A)
Again, we’ve chipped away at the null space some! But note: this applies when we know the inclination exactly. If we need to solve for it (or marginalize over it), we’ll obviously be dealing with extra uncertainty.
Limb darkening¶
The presence of limb darkening causes different parts of the map to be weighted differently at different phases. This breaks further symmetries in the problem:
[19]:
map = starry.Map(5, 1)
map.inc = 60
map[1] = 0.5
A = map.design_matrix(theta=theta)
compute_info(A)
Note, again, that above we assume we know the limb darkening law exactly, which is never the case in practice!
Transits and occultations¶
Transits or occultations break many symmetries. Let’s transit a small body across our map as it rotates, and look at the null space of the problem. We’ll have the body transit twice, occulting a different portion of the map each time (since it’s rotating).
[20]:
map = starry.Map(5, 1)
map.inc = 60
map[1] = 0.5
xo = 1.5 * np.sin(2 * np.pi * np.linspace(0, 2, len(theta)) + 0.5 * np.pi)
yo = 0.3
zo = 1.5 * np.cos(2 * np.pi * np.linspace(0, 2, len(theta)) + 0.5 * np.pi)
ro = 0.1
A = map.design_matrix(theta=theta, xo=xo, yo=yo, zo=zo, ro=ro)
compute_info(A)
The presence of an occultor has made a huge difference: the size of the null space is now nearly zero! Here’s what a light curve might look like for this situation:
[21]:
map.load("earth")
flux0 = A.dot(map.y)
sigma = 1e-3
flux = flux0 + sigma * np.random.randn(len(flux0))
plt.plot(theta, flux)
plt.xlabel("angular phase [deg]")
plt.ylabel("flux [normalized]");
You can see the small transits at angular phases of 150 and 300 degrees. Given this light curve, let’s add a Gaussian prior on the coefficients and linearly solve for the map:
[22]:
map.set_data(flux, C=sigma ** 2)
map.set_prior(L=1e-4)
map.solve(design_matrix=A)
map.show(projection="rect")
Here’s the true map for reference:
[23]:
map.load("earth")
map.show(projection="rect")
As expected, we did quite well! Although it’s blurry, we can pick out the major continents and oceans. The punchline: transits and occultations are great ways to break degeneracies. Stars transited by (multiple) planets or stars in eclipsing binary systems are therefore excellent targets for mapping.
Note
The example above is super idealized. The flux error was extremely small, we knew the inclination, limb darkening coefficients, and occultor parameters exactly, and we had transits across different portions of the surface. In most cases, things will not be this ideal, and the null space will be larger and/or the recovered maps will have much larger uncertainty.
Reflected light¶
The last thing we’ll touch on is phase curves in reflected light. Consider the idealized case of an inclined planet illluminated by a star with a rotational period that’s different from its orbital period:
[24]:
map = starry.Map(5, reflected=True)
map.inc = 60
xs = np.sin(2 * np.pi * np.linspace(0, 1.33, len(theta)))
ys = np.sin(2 * np.pi * np.linspace(0, 1.33, len(theta)))
zs = np.cos(2 * np.pi * np.linspace(0, 1.33, len(theta)))
A = map.design_matrix(theta=theta, xs=xs, ys=ys, zs=zs)
compute_info(A)
There is no null space! This example is highly contrived (the null space pops back up as soon as you increase the degree of the map, for instance) but it shows how powerful reflected light can be. The reason is that the day/night terminator breaks most of the symmetries in the problem, especially if it precesses relative to the rotational axis (as it does for an inclined planet).


X Factor fans rage as Martin Bashir is saved again and brand show a 'f***ing joke'
Victoria Ekanoye and ex-EastEnders star Jonny Labey were sent home from The X Factor tonight, leaving fans seething with rage
by Rose Hill, https://www.mirror.co.uk/authors/rose-hill/X Factor fans were left utterly furious as Martin Bashir was saved a second time by Simon Cowell - despite forgetting his lyrics to his song last night.
The journalist, 56, landed in the bottom three of the public vote with former Coronation Street star Victoria Ekanoye and ex-EastEnders actor Jonny Labey.
Victoria was immediately eliminated after receiving the fewest votes, meaning that it was up to judges Simon, Nicole Scherzinger and Louis Walsh to decide Martin and Jonny's fate.
Nicole and Louis decided to save their respective acts, however Simon refused to choose and left it to the public vote.
Host Dermot O'Leary revealed that Jonny would be leaving the competition, despite winning a safe seat last week.
The judges were left stunned, as were the contestants themselves.Martin quipped: "There's some kind of psychotic breakdown in the country at the moment!"
However viewers flocked to Twitter in anger, branding the show a "f***ing joke".
However viewers were furious and flocked to Twitter in anger.
One calmly tweeted: "WHY DO I BOTHER WATCHING THIS SHOW EVERY YEAR FOR THIS S*** #XFactorCelebrity."
While another commented: "WHAT THE F***! WHY ARE THEY SO OBSESSED WITH KEEPING MARTIN?!? #XFactorCelebrity #XFactor."
A third remarked: "Simon what are you playing at?! This is officially a farce, a f***ing joke.
Another wrote: "Seriously though first Olivia now Jonny... this is why nobody takes X Factor seriously lol #XFactorCelebrity."
One commented: "What the actual how did Martin get more votes than Jonny and Victoria #XFactorCelebrity."
"X FACTOR has just become the biggest joke. Yet again wrong act sent home. Simon I think you need to see a doctor #XFactorCelebrity," another fumed.
Earlier in the programme, Martin was close to tears in the middle of his performance to Lionel Richie's Easy.
Less than halfway through the song, he shocked everyone by announcing: "I've forgotten my words."
He tried to carry on but later added: "I do apologise."
Backstage, fellow star Victoria held her hands to her chest while Jenny looked on in horror.
Afterwards, Simon Cowell asked what had happened and Martin claimed that a last minute song change was to blame for his lapse in memory.
"I'm not trying to make excuses but my song had to change," he said. "I've been anxious all evening and it went out of my mind."
*The X Factor returns next Saturday on ITV


--- layout: post status: publish published: true title: Gonna have to call Alfonseca - Episode 8.21 author: display_name: Corey Fineran login: Corey email<EMAIL_ADDRESS>url: '' author_login: Corey author_email<EMAIL_ADDRESS>wordpress_id: 11821 wordpress_url: http://ivyenvy.com/?p=11821 date: '2016-05-15 22:18:09 +0000' date_gmt: '2016-05-16 03:18:09 +0000' categories: - Podcast tags: [] comments: - id: 23413 author: Rives Collins author_email: '' author_url: '' date: '2016-05-16 13:07:56 +0000' date_gmt: '2016-05-16 18:07:56 +0000' content: The new arrangement is GREAT -- bought mine this morning. I really hope we'll be hearing By the Lakeside a lot in 2016. Katie Day is the best! - id: 23460 author: jdcallister author_email: '' author_url: '' date: '2016-05-18 13:09:55 +0000' date_gmt: '2016-05-18 18:09:55 +0000' content: Any version on this song is great! Love it! Keep it up Katie. And GO CUBS!!! ---
Welcome to the 399th episode of The Ivy Envy Podcast. We are just one episode from #400!
On this episode, we talk about the Cubs series win over the Pittsburgh Pirates. There's a lot of positives to discuss, and apparently the Pirates players and fans:
Kurt shares a Jon Lester scary stat that is a good example of why you shouldn't pay attention to pitcher win-loss records.
On this episode, we premiere Katie Day's new version of her song, "By the Lakeside." Anyone that purchased this song through the embed below by Tuesday (May 17, 2016) night at midnight Pacific time will have their name entered into a drawing for a free Ivy Envy shirt.


{% load staticfiles %}
Want to Convert Your Programming Source Codes Into HTML Supported Format With Syntax Highlighting Features. Just Copy and Paste Your Source Codes In Input Box, Select Required Options And Press Highlight. Then, Click Into Output Box, Copy All Syntax And Paste it in any HTML Supported Platform, Blog, Email, Wordpress, Website And much more.


CWP No. 18381 of 2016(O&M) [1] IN THE HIGH COURT OF PUNJAB AND HARYANA AT CHANDIGARH. CWP No. 18381of 2016(O&M) Date of decision : 30.01.2017 Sanjeev Kumar ......Petitioner Versus State of Punjab and others ........Respondents CORAM : HON'BLE MR. JUSTICE S.S.SARON HON'BLE MR. JUSTICE DARSHAN SINGH Present: Mr. Dinesh Ghai, Advocate for the petitioner. Mr. P.P.S.Thethi, Addl.AG, Punjab for respondent no.3. **** S.S.SARON, J.
Petitioner-Sanjeev Kumar by way of the present petition under Articles 226 and 227 of the Constitution of India seeks quashing of the order dated 12.07.2016 (Annexure P-12) passed by the Sr. Environmental Engineer, Punjab Pollution Control Board, Zonal Office, Patiala (respondent no.2) whereby the No Objection Certificate (NOC) No. G16PTACTE410442 dated 24.05.2016 issued to the petitioner vide office letter dated 25.05.2016 (Annexure P-2) has been revoked/cancelled.
The petitioner purchased land measuring 22 Bighas 4 Biswas at village Shambu Kalan, Tehsil Rajpura, District Patiala in the name of 'M/s A- One Poultry Farm' vide registered sale deed dated 16.03.2016 (Annexure P-1). The petitioner had already sought permission/NOC from the State of Punjab 1 of 8 ::: Downloaded on - 09-07-2017 10:06:39 ::: CWP No. 18381 of 2016(O&M) [2] through its Member Secretary, Punjab State Pollution Control Board, Vatavaran Bhawan, Nabha Road, Patiala (respondent no.1) ('Board' - for short) to establish and start business/industry of poultry farm in the name and style of 'M/s A-One Poultry Farm'. The permission was granted to the petitioner vide NOC G16PTACTE410442 dated 24.05.2016 (Annexure P-2), which was issued vide letter dated 25.05.2016. The petitioner got the construction of infrastructure started in the poultry farm in question. According to the petitioner, the construction had been got stopped by notorious persons as they did not want the poultry farm to continue probably for their own self motive. Therefore, threats were held out to the petitioner and also to the labour working at the site of construction. The matter was also reported to the concerned police post in terms of a complaint dated 17.06.2016 (Annexure P-3). Since, the threats had continued, the petitioner filed CRM-M-23398 of 2016 titled as "Sanjeev Kumar Vs. The Sr. Superintendent of Police, Patiala and others" in this Court for providing protection to his life and liberty. The said petition was disposed of by this Court vide order dated 22.07.2016 (Annexure P-4) directing the Senior Superintendent of Police, Patiala to look into the contents of the complaint dated 17.6.2016 submitted to the concerned police in-charge and to take necessary action thereon in accordance with law. According to the petitioner, notorious persons who belong to the ruling party/State Govt. had their personal interests and had got the construction of the poultry farm stopped.
Notice of motion and notice regarding stay was issued to the respondents.
Mr. P.P.S.Thethi, Additional Advocate General, Punjab has filed reply by way of affidavit of Mr. Harpreet Singh Sudan, IAS, Sub Divisional Magistrate, Rajpura, District Patiala (Respondent no.3).
2 of 8
::: Downloaded on - 09-07-2017 10:06:40 :::
CWP No. 18381 of 2016(O&M) [3]
The said reply is taken on record.
A short reply of Sh. Rakesh Kumar Nayar, Environment Engineer of the Board has already been filed on behalf of respondents No.1 and 2. According to the said reply a complaint was received from the office of the Sub-Divisional Magistrate, Rajpura regarding nuisance caused by poultry farms existing at villages Bathonia Kalan, Bathonia Khurd, Suhron and Jakhepal etc. The Sub Divisional Magistrate, Rajpura, it is submitted, called a meeting on 01.04.2016 and it was decided that the Board Officers would visit six poultry farms at the aforesaid villages along with representatives of the villages and would submit their report to the Sub-Divisional Magistrate. The poultry farms were visited on 01.04.2015 (sic. 2016) and again on 02.05.2016 and detailed reports of the Environmental Engineer of the Board were sent to the Sub-Divisional Magistrate, Rajpura. Meanwhile, residents of villages gathered at village Bathonia and started agitation against the authorities demanding that the poultry farms be shut down immediately. The agitation of the villages continued for more than a week. The roads leading to the poultry farms were blocked. The District Administration made different committees to assess the situation and also take preventive measures to control the nuisance caused from the operation of the poultry farms.
It is further submitted that the petitioner had applied for 'consent to establish' (NOC) from the pollution angle for the establishment of poultry farm for 1,00,000 birds (Eggs @ 70,000 nos./day) at village Shambu Kalan, Tehsil Rajpura, District Patiala. The poultry farm submitted the requisite documents and fee along with its application form. The site was visited by an officer of the Board on 16.05.2016. The proposed site met with the general guidelines to establish the unit as per the Board's policy dated 30.04.2013 and environmental guidelines for poultry farm issued by CPCB vide letter dated 3 of 8 ::: Downloaded on - 09-07-2017 10:06:40 ::: CWP No. 18381 of 2016(O&M) [4] 20.10.2015. Therefore, after considering the application of the petitioner, 'consent to establish' (NOC) from pollution angle with certain conditions was granted vide letter 25.05.2016.
The Board then received a letter dated 06.06.2016 (Annexure R-2/1) from the Sub-Divisional Magistrate, Rajpura stating that in case the consent was granted for setting up a poultry farm, it would lead to a law and order situation as the residents of the village/Panchayat had expressed their disapproval for setting up a new poultry farm in the area and near to the land owned by the Gurudwara Sri Guru Granth Sahib. Therefore, it was addressed to the Board that the 'consent to establish' should not be granted to the poultry farm without the permission of the residents of the village and the Gram Panchayat. In the light of the aforesaid letter, the competent authority of the Board decided to revoke the NOC already granted to the poultry farm in terms of the advise of the Sub-Divisional Magistrate, Rajpura because there was public resentment against the establishment of the poultry farm. It is also mentioned that vide communication dated 11.11.2016 (Annexure R-2/2) the Sub Divisional Magistrate again informed the Board that in case permission was granted for the establishment of a poultry farm, it would lead to serious law and order situation. The 'consent to establish' (NOC) granted to the poultry farm had been cancelled by the Board keeping in view the recommendations/directions of the Sub-Divisional Magistrate, Rajpura in order to avoid a serious law and order situation in the said area. It is, however, submitted that the Board had no objection in granting the NOC if the District Administration has no reservation in this regard.
In terms of the reply of respondent No.3, it is stated that the petitioner after obtaining NOC from the Sr. Environmental Engineer of the Board (respondent no.2), started construction work at the site at village 4 of 8 ::: Downloaded on - 09-07-2017 10:06:40 ::: CWP No. 18381 of 2016(O&M) [5] Shambu Kalan, Tehsil Rajpura, District Patiala. However, residents of the neighboring village as also the Gram Panchayat raised protests against the said construction. The predecessor of the present Sub Divisional Magistrate, Rajpura vide letter dated 06.06.2016 (Annexure R-3/1) sent a request accompanied by applications of the Gram Panchayat of villages Basmah, Sharnbu Kalan, Shambu Rajputa, Lohad, Nandgarh, Sarai Mugal, Ghadama Kalan/Khurd, Nosahara, Ram Nagar Sainiya, Nanhera, Tepla, Ragarh and Gurudwara Sahib Shambu Kalan to the State Government through Member Secretary of the Board and the Senior Environmental Engineer of the Board (respondents no.1 and 2). It was submitted that 'M/s A-One Poultry Farm' Shambu Kalan wanted to start a poultry farm in the land in Khewat Khata No.620/753 and 804/944. The said land was in the ownership share and possession of Guru Granth Sahib, Shambu Kalan. Its boundary ridge ('watt') adjoins the land of the Gurudwara Sahib. In case the petitioner succeeds in setting up a poultry farm in the said land, then there would be great difficulty with respect to the above referred Gurudawara Sahib and other residents of village Shambu Kalan. The Gram Panchayat of Village Shambu Kalan had also passed a resolution dated 17.05.2016 for stopping the above referred poultry farm to be operational. It is submitted that in case, without the consent of the Panchayat, the poultry farm was allowed in the vicinity of the adjoining Gurudwara Sahib, then feelings of the people would be hurt and there would be danger to law and order. The Sub-Divisional Magistrate, Rajpura (respondent No.3) sought a fresh report regarding law and order from the Deputy Superintendent of Police, Ghanour vide letter dated 14.10.2016 (Annexure R-3/2). This was sought after letter dated 06.10.2016 (Annexure R-3/3) was received from the Environmental Engineer of the Board (respondent No.2). The Deputy Superintendent of Police, Ghanour vide letter 5 of 8 ::: Downloaded on - 09-07-2017 10:06:40 ::: CWP No. 18381 of 2016(O&M) [6] dated 10.11.2016 (Annexure R-3/4) submitted if the construction of M/s A- One Poultry Farm was started or it was allowed to function, then the residents of the area would raise strong protests and there would be an apprehension of breach of peace and law and order.
In short, the case of the petitioner is that for setting up a poultry farm by the name of M/s A-One Poultry Farm in village Shambu Kalan, he was granted 'consent to establish' an industrial unit under Section 24 of the Water (Prevention and Control of Pollution) Act, 1974 and Section 21 of the Air (Prevention and Control of Pollution) Act, 1981 vide communication dated 24.05.2016 (Annexure P-2). The said consent was granted by the Sr. Environmental Engineer of the Board (respondent no.2). However, now the said consent had been revoked vide letter dated 12.07.2016 (Annexure P-12). The said revocation of the licence of the petitioner is without hearing or even issuing a show cause notice to him.
We have given our thoughtful consideration to the matter. We are of the view that the rule of audi alteram partem is a well accepted facet of the principles of natural justice as enshrined by the provisions of Article 21 of the Constitution of India and a person has a right to be heard before any order adverse to him and affecting his rights is passed.
In National Textile Workers' Union etc. v. P.R.Ramakrishnan and others, AIR 1983 SC 75 it was observed that the audi alteram partem rule which mandates that no one shall be condemned unheard is one of the basic principles of natural justice and if this rule is held to be applicable in a quasi- judicial or even in an administrative proceeding involving adverse civil consequences, it would a fortiori apply in a judicial proceeding such as a petition for winding up of a company. It was further observed that it would indeed be strange that the workers who had contributed to the building of the 6 of 8 ::: Downloaded on - 09-07-2017 10:06:40 ::: CWP No. 18381 of 2016(O&M) [7] enterprise as a centre of economic power should have no right to be heard when it is sought to demolish that centre of economic power.
In the circumstances, no order involving adverse civil consequences is liable to be passed against any person without giving him an opportunity to be heard against the order that is to be passed and this rule of natural justice applies to quasi-judicial and administrative proceedings. The Senior Environmental Engineer of the Board (respondent no.2), in the present case, passed the order dated 12.07.2016 (Annexure P-12) without notice and without hearing the petitioner. The Senior Environment Engineer of the Board (respondent no.2) who passed the impugned order may or may not be justified in doing so; however, the right of hearing to an aggrieved person, who has got the NOC/consent to operate the poultry farm after complying with all the necessary conditions was not liable to be dispensed with. Therefore, the petitioner was not liable to be non-suited on the ground as had been stated in the impugned order (Annexure P-12). We are of the view that the order dated 12.07.2016 (Annexure P-12) is to be invalidated and an opportunity of hearing is liable to be given to the petitioner so as to explain his position and satisfy the authorities of his case.
Accordingly, the civil writ petition is allowed and the order dated 12.07.2016 (Annexure P-12) is set aside and quashed and the matter is remitted back to the Board for passing a fresh order after hearing the petitioner.
Learned counsel for the petitioner states that till such time the matter is heard and decided, he shall maintain status quo at the site. However, it is prayed that the Board may expedite the hearing.
In the circumstances, status quo with regard to the position at the site shall be maintained by the petitioner. However, the Board shall consider and decide the case of the petitioner as expeditiously as possible and 7 of 8 ::: Downloaded on - 09-07-2017 10:06:40 ::: CWP No. 18381 of 2016(O&M) [8] preferably within three months from the receipt of the copy of the order. It is also submitted by the learned counsel for the petitioner that the NOC is up till May, 2017 and in case the NOC is to be continued, the same may also be considered for extension of period.
The Board at the time of consideration of the case of the petitioner for grant of consent to operate in case it decides to grant the necessary consent, it shall also consider the extension of time to the petitioner to a date beyond May 2017, keeping in view the present litigation.
(S.S.SARON)
JUDGE
(DARSHAN SINGH)
January 30, 2017 JUDGE
s.khan
Note:
Whether the order is speaking/reasoned : Yes/No
Whether the order is Reportable : Yes/No
8 of 8
::: Downloaded on - 09-07-2017 10:06:40 :::


BERKSHIRE HATHAWAY INC.
3555 Farnam Street Suite 1440
Omaha, NE 68131
Offical Home Page
FOR A FREE CAR INSURANCE RATE QUOTE THAT COULD SAVE YOU SUBSTANTIAL MONEY
WWW.GEICO.COM OR CALL 1-888-395-6349, 24 HOURS A DAY
If you have any comments about our WEB page, you can either write us at the address show above or e-mail us at<EMAIL_ADDRESS>However, due to the limited number of personnel in our corporate office, we are unable to provide a direct reponse.


HARD RAIN (1998)
A film review by Steve Kong Copyright 1998 Steve Kong
Ding! Here's movie number two for my top 10 worst films of 1998 list! I'm building quite a collection: first fire (Firestorm), now water. I wonder what other natural wonders January will bring me.
Scripted by Graham Yost, Hard Rain follows Yost's formula: high concept, low characterizations, and even lower plot. Yost likes to write chase movies, which is OK, but he tends to write them so formulaic that it's getting tired now, so please Graham come up with something new. Let me give you a run down of the films that Yost has written so far: Speed, bomb on bus, chase down time trying to stop the bomb. Worked well because of, then rookie cinematographer-turned-director, Jan De Bont's excellent eye for action and tension.  Worked well because of John Woo's style and excellent eye for action. Hard Rain, stolen money, must chase down money because the bad guys are greedy. Doesn't work because of the eye of cinematographer-turned-director Mikael Salomon who muddles up the action and can't build tension.
Yost, as I pointed out in my Broken Arrow review, needs to have a good director working on his scripts. If not the scripts fall apart because the audience is given too much time to think about how ludicrous his scripts are and because the audience is not convincingly wowed by the spectacular action. We are given more than enough time to think about Yost's script in Hard Rain. Hard Rain runs a few minutes over an hour and a half, and it has got to be the longest, and most painful, hour and a half -- next to Firestorm -- that I've spent in a theatre in the last few months.
Here's a run down of the premise of the story. During a huge storm in Huntingburg, Indiana, a group of men lead by Jim (Morgan Freeman) tries to steal a cool three million dollars from an armor truck. Charlie (Ed Asner), with Tom (Christian Slater) riding along, is driving the armored truck. After a shootout, Charlie is dead, Tom has hidden the money, and the chase is on. Throw in a greedy town sheriff (Randy Quaid) and a damsel-in-distress (Minnie Driver) and this is what Yost calls a good script.
Next to a sudden twist near the end of the film, this film is very predictable and boring. The action is bland and when it is not bland, rain and some bad camera work muddle it.
Mikael Salomon, who's biggest stint so far was as cinematographer for James Cameron on The Abyss tries hard to make this film work, but the film is weighted down by a few factors. The biggest would be the script. Along with this would be the acting, next to Morgan Freeman, who is always great onscreen, there are no other good performances. The other performances range from bland (Minnie Driver) to wimpy (Christian Slater) to silly (Randy Quiad). There is too much "padding-time." Padding-time is what I consider time onscreen when there is really nothing except to pad the movie's running time to the requisite 90 minutes or more. Yost fills this padding-time with silly one-liners and two old folk (Rick Harris and Betty White) for comedy relief. Talking about one-liners, Yost doesn't seem to have a concept about conversation or dialogue. Whenever a character onscreen opens their mouth, it seems to be either to scream or spit out a one-liner. And lastly, Christian Slater as Tom just does not work. Ever since I saw Christian Slater try to become an action hero in Broken Arrow I was convinced that he couldn't. And he proves it in Hard Rain. Slater is just a wimp.
The studio-heads change the name of this film from The Flood to Hard Rain after it performed badly with test audiences. My suggestion to the studio-heads: should have change a lot more or kept it in the vault. Skip this film, it's awful.
--- steve kong<EMAIL_ADDRESS>find the latest movie reviews at the hard boiled movie page http://boiled.sbay.org/boiled/
The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


Title: A day's worth of parking in Chester Square
Brian Fogarty, Ysabelle Kempe, Lal Birali, Victoria Yang
Service-Learning Course Project as part of DS 4200: Information Presentation and Visualization, taught by Prof. Cody Dunne, Data Visualization @ Khoury, Northeastern University.
This project was developed in conjunction with the Chester Square Neighborhood Association. The goal of the Neighborhood Association is to show that there is a problem with parking availability and distribution in the area. Residents are often unable to find parking close to their homes, and there are complaints of inefficient parking practices.
The Neighborhood Association wants to bring the problem of parking to the attention of local government. The Association already knows there is a problem and is able to explain the issue and potential causes in words (e.g., not enough parking, residents don’t know where to park, etc.), but they want to be able to show numbers that prove their point.
We hope this project serves as a valuable tool for the Neighborhood Association to communicate the need for change to authorities. Cars are a key player in daily urban life, and parking needs to be taken seriously in order to create a better standard of living for those on both sides of Chester Square.
When designing our project, we asked ourselves how occupied Chester Square parking is throughout the day. We wanted our final piece to show change over time, as well as give users an idea of how difficult it is to find parking at any given time. Through our own research and data collection, we have concluded that a reasonable response would be to more clearly mark parking spots, as well as conduct another survey.
Here is our final visualization. Use the slider to select the time of day shown. Hover over the bar chart to see more details on empty and occupied spots.
Our visualization includes an interactive map showing occupancy rate, as well a bar chart that shows the empty and occupied parking spots on each street. The user is able to use a slider to change the time of day, which is reflected in the visualizations.
For our map, we used color to represent occupancy rate. We used a color scale of red, white, and green because we believed it was intuitive for red to mean that there are less spots available and green to mean there are more spots available. We designed the map to only show streets because we didn't want to clutter the visualizations with buildings or other landmarks.
We included the bar chart as well because it is very important for a user not only to know the proportion of spots filled, but to know exactly how many spots are left. We used a simple stacked bar chart to show this data, which uses area to show the parking capacity of a certain street, as well as area and color to show the breakdown of empty versus occupied spots.
For interactivity, users can use the slider to update both visualizations, as well as hover over the bar chart for more detailed information on the number of empty and occupied spots.
We'd also like to credit John Walley's Simple Slider sample which helped us a lot.
Below, you can see screenshots of our final visualization.
Below, you will find links to our presentation slides.
Our data was collected during a parking survey conducted in Chester Square from 6 a.m. to 8 p.m. on Oct. 29, 2019.
Filled spots, empty spots, and blocked spots were recorded by hand on the hour. We filled this data into a Google Sheet by hand. This original Google Sheet included each spot with a unique id number (integer), the street (string), and the parking status for each spot throughout each hour of the data collection. If a car was parked in a spot, we used the first three characters of the license plate (string). If the spot was blocked, we used "blocked". If there was construction blocking the spot, we used the value "construction". If there was simply no car parked in the spot, we did not include a value in that cell.
For the bar chart data, we used Python to clean and preprocess. We iterated through the data and aggregated the number of occupied spots and full capacity for each street. The final data has street name as a string, occupied spots as an integer, and capacity as an integer for each time of day.
For the map data, we also used Python with the NumPy and Pandas libraries to clean and preprocess. The output was a csv file with polygon coordinates (floats), occupancy counts (integers), occupancy rates (floats), and percentages (floats).
| Index | Domain Task | Analytic Task | Search Task | Analyze Task |
| 1 | What are the occupancy rates at different times of the day? | Identify | Browse | Discover |
| 2 | Where are the problem areas for after-work parking? | Identify | Browse | Present |
| 3 | What is the average duration a single vehicle is parked? | Identify | Lookup | Present |
| 4 | Who are the owners of vehicles parked in the SA? | Identify | Browse | Discover |
| 5 | How do the various parking regulations affect occupancy and turnover rates? | Summarize | Explore | Present |
| 6 | What are the turnover rates for different times of day? | Compare | Browse | Present |
This task table shows that our main goal is to understand how the occupancy rates fluctuate throughout the day. It is also to understand problem areas for parking. What are the areas that most frequently have no available spaces? By understanding where the problem areas are and what time of the day issues arise, we can help the Neighborhood Association prioritize certain areas for improvement.
The final results of our project are geared toward presentation and discovery. We want the Association to be able to take this visualization and present its findings to community members or local government figures. In order to achieve our task goals, we knew this visualization had to be simple, uncluttered, and easily understood by individuals without a data science background. We wanted it to tell the story of a day of parking in Chester Square, which is partially why we used the recognizable shape of Chester Square in our map.
Our top two tasks both had to do with identifying and browsing, so we decided to create a visualization that would answer both of those questions. We abstracted these tasks together to create the new task goal of understanding the number/roportion of free spots throughout the day.
Below, you can see our initial visualization draft. Originally, we had imagined the color scale as red, yellow, and green. Spatial position is one of the most valuable encodings in data visualization, and we wanted to use a map to show where exactly the problem areas are for parking. We simplified the map down to just streets (rather than including landmarks and buildings) because we wanted to avoid chart clutter. In this original sketch, we imagined that the bar chart on the bottom would show the number of available spots in the area at each hour. We did this because we wanted our vis to be very practical and usable in the goal of showing people how easy it is to find parking at a specific time. Color coding on this bar chart would have corresponded to the color scale for the map, with green meaning more spots are available and red meaning less spots are available. A slider in this sketch controls the time of day currently shown on the map. For each selected time of day, that bar on the bar chart would have been highlighted.
Our next visualization sketch, shown below, was done digitally using Tableau and Adobe Illustrator. We made a few very substantial changes in this iteration. One of these changes was to alter our color scale from red, yellow, and green to red, white, and green. We belive that this color scale is more intuitive because some people may not associate yellow with a midpoint between green and red. White is more neutral. We toyed with the idea of showing individual cars on our map rather than occupancy rates, but we decided against it because we wanted to keep the vis simple enough for users to gain a broader main takeaway. We altered our bar chart to a stacked bar chart that shows the number of occupied and empty spots for each street, rather than for each time of day. As the slider updates the time shown on the map, the stacked bar chart also updates. This makes our visualization more information-dense without too much added complexity. We nixed color on the bar chart because we did not feel it was intuitive to have the same color scale for both visualization portions since they are showing different aspects of our data. We added a filter option for users to choose which types of regulation spots they wanted to see. This would help make the visualization customizable, more granular, and show varying levels of complexity.
Our final visualization contains our planned map and bar chart, with the slider to update time of day. One aspect of our plan that we were not able to implement due to time constraints was filtering based on regulation. However, we still believe that this visualization gives a good overview of a day of parking in Chester Square. See below for screenshots of our final visualization.
Unfortunately, our group was unable to take part in the in-class usability testing. In fact, we hit quite a few roadblocks to creating our final product.  In other words, we had to cut some features of the visualization that, given more time, we would have included.
Parking is an extraordinarily important part of urban life. It allows people to own cars and have greater freedom of transportation. But, especially in a city like Boston, real estate is incredibly valuable and cars take up a lot of space.
This parking survey and the following vis was designed to identify parking problem areas in Chester Square, analyze how efficiently people park, and prompt ideas for improving parking in the neighborhood. When we began this project, none of us had any experience with parking analysis or parking surveys, but throughout the course of this project we began to understand how bad parking or bad parking spot design can have resonating effects on the layout of a neighborhood. Parking availability can vastly effect quality of life for residents because it could mean the difference between going straight home after a long day at work or having to trudge for 10 minutes in the cold darkness to get home.
Through our experiences walking around Chester Square, as well as our data analysis, we discovered that there are parking issues in the neighborhood. There were many people who parked very far away from eachother, but did not leave enough space for another car to park. A suggestion we have is for local government to add painted parking spots onto the street so that people are directed to park more efficiently.
For further analysis, we would also recommend a follow-up parking survey be performed on a day with more ideal conditions. We were forced to work on a rainy day when there was construction, which skewed our results.
List here where any code, packages/libraries, text, images, designs, etc. that you leverage come from.


--- layout: newsstory title: STEADY AS SHE GOES match: Yeltz 2 Redditch 1 ---
The mighty Yeltz maintained their 100% home record so far with a 2-1 victory against Redditch . We dominated the first half, with goals from Neil Smith and a first from Leroy May . They pulled one back through a long range shot over Tim Clarke, but we held on for a welcome three points.
We've moved up to 6th place in the table , just 5 points behind early leaders Solihull Borough. (I know this keeps changing - Chas and Darren keep correcting my stats!) Not too bad a start with a team that maybe needs a little time to gel.
On Saturday we start the long road to Cardiff (honest) with an F.A.Cup Preliminary Round tie at home to Chasetown . Another consequence of getting relegated is losing our bye through to the 2nd Qualifying Round, but hopefully the draw has been reasonably kind to us. It would be really nice to get some sort of run going in the Cup again.
Halesowen Town: Clarke; Colwell, Crawford, Smith, Collins(c), Burnham, Hall (James 79), Giddings, May, Spencer, Hines. Subs: Gavillet, Martin.
Redditch Utd: Willetts; Mitchell, Bradley, Field(c) (Barratt 89), Creighton, Frost, Swann, Cartwright, Danks, Began, Colcombe. Subs: Hackett, Williams.
Booked: Frost (34), Giddings (34), Hall (38), Field (40), Clarke (51), Bradley (69), Crawford (85), Swann (85) Referee: J. Bottomer (Chislehurst) Attendance: 478
In the end the result of the game looked better than the means by which it was achieved - though allowances should be made for Halesowen missing five key players - Andy Jones, Mark Taylor, Stuart Skidmore, Ross Collins and Lee Knight. Hopefully, this early season disruption should start to rectify itself shortly.
For the third time in four matches, Halesowen took an early lead within five minutes of the start when a Les Hines freekick found Leroy May and Neil Smith in the clear, but it was Smith's head that claimed the goal. Andy Spencer then hit Colwell's cross against the underside of the bar in the 13th minute with the visitors looking ragged.
Redditch boasted ex-Halesowen assistant manager Gary Hackett on the bench - in charge due to the inexplicable absence of manager Nicky Cross. On the pitch Redditch also had ex-Halesowen players Neil Cartwright, Andy Bradley and Mark Creighton all seeking to impress against their old club.
It took Halesowen until the 45th minute to increase the lead when Leroy May powered home a diving header from another Les Hines set-piece - this time a right wing corner.
Many thought Halesowen would go on to record a thumping win but it was not to be and in the end the final whistle of Mr Bottomer, keen to get back home to Kent, strangely enough, came as somewhat of a relief.
As in previous matches Halesowen collected a handful of yellow cards, but this time were fortunate not to have a player dismissed. The referee judged Tim Clarke had handled the ball out of his area in the 51st minute when trying to save from an on-rushing striker, but didn't carry through the letter of the law by making Clarke (and Halesowen) suffer the inevitable consequences. Indeed, it appeared Les Hines was already preparing to take charge in goal whilst Mr. Bottomer was delivering his punishment to Clarke. In the 85th minute Michael Crawford, who looked out of sorts at left back, tussled with Mark Swann and as they were both walking over for a lecture from the referee, Crawford aimed a head-butt at Swann. Even though no contact appeared to have been made, Crawford can consider himself lucky to escape with a "yellow".
Redditch pulled a goal back to change the course of the game in the 58th minute when Swann volleyed home after playing a one-two with Scott Colcombe. From then on, the game degenerated into a scrappy affair, but Halesowen had just enough in hand to hold onto the three points.


The default configuration parameters allow Flink to run out-of-the-box in single node setups.
This page lists the most common options that are typically needed to set up a well performing (distributed) installation. In addition a full list of all available configuration parameters is listed here.
All configuration is done in conf/flink-conf.yaml
, which is expected to be
a flat collection of YAML key value pairs
with format key: value
.
The system and run scripts parse the config at startup time. Changes to the configuration file require restarting the Flink JobManager and TaskManagers.
env.java.home
: The path to the Java installation to use (DEFAULT: system's
default Java installation, if found). Needs to be specified if the startup
scipts fail to automatically resolve the java home directory. Can be specified
to point to a specific java installation or version. If this option is not
specified, the startup scripts also evaluate the $JAVA_HOME
environment variable.
jobmanager.rpc.address
: The IP address of the JobManager, which is the
master/coordinator of the distributed system (DEFAULT: localhost).
jobmanager.rpc.port
: The port number of the JobManager (DEFAULT: 6123).
jobmanager.heap.mb
: JVM heap size (in megabytes) for the JobManager
(DEFAULT: 256).
taskmanager.heap.mb
: JVM heap size (in megabytes) for the TaskManagers,
which are the parallel workers of the system. In
contrast to Hadoop, Flink runs operators (e.g., join, aggregate) and
user-defined functions (e.g., Map, Reduce, CoGroup) inside the TaskManager
(including sorting/hashing/caching), so this value should be as
large as possible (DEFAULT: 512). On YARN setups, this value is automatically
configured to the size of the TaskManager's YARN container, minus a
certain tolerance value.
taskmanager.numberOfTaskSlots
: The number of parallel operator or
UDF instances that a single TaskManager can run (DEFAULT: 1).
If this value is larger than 1, a single TaskManager takes multiple instances of
a function or operator. That way, the TaskManager can utilize multiple CPU cores,
but at the same time, the available memory is divided between the different
operator or function instances.
This value is typically proportional to the number of physical CPU cores that
the TaskManager's machine has (e.g., equal to the number of cores, or half the
number of cores).
parallelization.degree.default
: The default degree of parallelism to use for
programs that have no degree of parallelism specified. (DEFAULT: 1). For
setups that have no concurrent jobs running, setting this value to
NumTaskManagers * NumSlotsPerTaskManager will cause the system to use all
available execution resources for the program's execution.
fs.hdfs.hadoopconf
: The absolute path to the Hadoop File System's (HDFS)
configuration directory (OPTIONAL VALUE).
Specifying this value allows programs to reference HDFS files using short URIs
(hdfs:///path/to/files
, without including the address and port of the NameNode
in the file URI). Without this option, HDFS files can be accessed, but require
fully qualified URIs like hdfs://address:port/path/to/files
.
This option also causes file writers to pick up the HDFS's default values for block sizes
and replication factors. Flink will look for the "core-site.xml" and
"hdfs-site.xml" files in teh specified directory.
taskmanager.tmp.dirs
: The directory for temporary files, or a list of
directories separated by the systems directory delimiter (for example ':'
(colon) on Linux/Unix). If multiple directories are specified, then the temporary
files will be distributed across the directories in a round-robin fashion. The
I/O manager component will spawn one reading and one writing thread per
directory. A directory may be listed multiple times to have the I/O manager use
multiple threads for it (for example if it is physically stored on a very fast
disc or RAID) (DEFAULT: The system's tmp dir).
jobmanager.web.port
: Port of the JobManager's web interface (DEFAULT: 8081).
fs.overwrite-files
: Specifies whether file output writers should overwrite
existing files by default. Set to true to overwrite by default, false otherwise.
(DEFAULT: false)
fs.output.always-create-directory
: File writers running with a parallelism
larger than one create a directory for the output file path and put the different
result files (one per parallel writer task) into that directory. If this option
is set to true, writers with a parallelism of 1 will also create a directory
and place a single result file into it. If the option is set to false, the
writer will directly create the file directly at the output path, without
creating a containing directory. (DEFAULT: false)
taskmanager.network.numberOfBuffers
: The number of buffers available to the
network stack. This number determines how many streaming data exchange channels
a TaskManager can have at the same time and how well buffered the channels are.
If a job is rejected or you get a warning that the system has not enough buffers
available, increase this value (DEFAULT: 2048).
taskmanager.memory.size
: The amount of memory (in megabytes) that the task
manager reserves on the JVM's heap space for sorting, hash tables, and caching
of intermediate results. If unspecified (-1), the memory manager will take a fixed
ratio of the heap memory available to the JVM, as specified by
taskmanager.memory.fraction
. (DEFAULT: -1)
taskmanager.memory.fraction
: The relative amount of memory that the task
manager reserves for sorting, hash tables, and caching of intermediate results.
For example, a value of 0.8 means that TaskManagers reserve 80% of the
JVM's heap space for internal data buffers, leaving 20% of the JVM's heap space
free for objects created by user-defined functions. (DEFAULT: 0.7)
This parameter is only evaluated, if taskmanager.memory.size
is not set.
These parameters configure the default HDFS used by Flink. Setups that do not
specify a HDFS configuration have to specify the full path to
HDFS files (hdfs://address:port/path/to/files
) Files will also be written
with default HDFS parameters (block size, replication factor).
fs.hdfs.hadoopconf
: The absolute path to the Hadoop configuration directory.
The system will look for the "core-site.xml" and "hdfs-site.xml" files in that
directory (DEFAULT: null).fs.hdfs.hdfsdefault
: The absolute path of Hadoop's own configuration file
"hdfs-default.xml" (DEFAULT: null).fs.hdfs.hdfssite
: The absolute path of Hadoop's own configuration file
"hdfs-site.xml" (DEFAULT: null).The following parameters configure Flink's JobManager and TaskManagers.
jobmanager.rpc.address
: The IP address of the JobManager, which is the
master/coordinator of the distributed system (DEFAULT: localhost).jobmanager.rpc.port
: The port number of the JobManager (DEFAULT: 6123).taskmanager.rpc.port
: The task manager's IPC port (DEFAULT: 6122).taskmanager.data.port
: The task manager's port used for data exchange
operations (DEFAULT: 6121).jobmanager.heap.mb
: JVM heap size (in megabytes) for the JobManager
(DEFAULT: 256).taskmanager.heap.mb
: JVM heap size (in megabytes) for the TaskManagers,
which are the parallel workers of the system. In
contrast to Hadoop, Flink runs operators (e.g., join, aggregate) and
user-defined functions (e.g., Map, Reduce, CoGroup) inside the TaskManager
(including sorting/hashing/caching), so this value should be as
large as possible (DEFAULT: 512). On YARN setups, this value is automatically
configured to the size of the TaskManager's YARN container, minus a
certain tolerance value.taskmanager.numberOfTaskSlots
: The number of parallel operator or
UDF instances that a single TaskManager can run (DEFAULT: 1).
If this value is larger than 1, a single TaskManager takes multiple instances of
a function or operator. That way, the TaskManager can utilize multiple CPU cores,
but at the same time, the available memory is divided between the different
operator or function instances.
This value is typically proportional to the number of physical CPU cores that
the TaskManager's machine has (e.g., equal to the number of cores, or half the
number of cores).taskmanager.tmp.dirs
: The directory for temporary files, or a list of
directories separated by the systems directory delimiter (for example ':'
(colon) on Linux/Unix). If multiple directories are specified, then the temporary
files will be distributed across the directories in a round robin fashion. The
I/O manager component will spawn one reading and one writing thread per
directory. A directory may be listed multiple times to have the I/O manager use
multiple threads for it (for example if it is physically stored on a very fast
disc or RAID) (DEFAULT: The system's tmp dir).taskmanager.network.numberOfBuffers
: The number of buffers available to the
network stack. This number determines how many streaming data exchange channels
a TaskManager can have at the same time and how well buffered the channels are.
If a job is rejected or you get a warning that the system has not enough buffers
available, increase this value (DEFAULT: 2048).taskmanager.network.bufferSizeInBytes
: The size of the network buffers, in
bytes (DEFAULT: 32768 (= 32 KiBytes)).taskmanager.memory.size
: The amount of memory (in megabytes) that the task
manager reserves on the JVM's heap space for sorting, hash tables, and caching
of intermediate results. If unspecified (-1), the memory manager will take a fixed
ratio of the heap memory available to the JVM, as specified by
taskmanager.memory.fraction
. (DEFAULT: -1)taskmanager.memory.fraction
: The relative amount of memory that the task
manager reserves for sorting, hash tables, and caching of intermediate results.
For example, a value of 0.8 means that TaskManagers reserve 80% of the
JVM's heap space for internal data buffers, leaving 20% of the JVM's heap space
free for objects created by user-defined functions. (DEFAULT: 0.7)
This parameter is only evaluated, if taskmanager.memory.size
is not set.jobclient.polling.interval
: The interval (in seconds) in which the client
polls the JobManager for the status of its job (DEFAULT: 2).taskmanager.runtime.max-fan
: The maximal fan-in for external merge joins and
fan-out for spilling hash tables. Limits the number of file handles per operator,
but may cause intermediate merging/partitioning, if set too small (DEFAULT: 128).taskmanager.runtime.sort-spilling-threshold
: A sort operation starts spilling
when this fraction of its memory budget is full (DEFAULT: 0.8).jobmanager.web.port
: Port of the JobManager's web interface that displays
status of running jobs and execution time breakdowns of finished jobs
(DEFAULT: 8081).jobmanager.web.history
: The number of latest jobs that the JobManager's web
front-end in its history (DEFAULT: 5).These parameters configure the web interface that can be used to submit jobs and review the compiler's execution plans.
webclient.port
: The port of the webclient server (DEFAULT: 8080).webclient.tempdir
: The temp directory for the web server. Used for example
for caching file fragments during file-uploads (DEFAULT: The system's temp
directory).webclient.uploaddir
: The directory into which the web server will store
uploaded programs (DEFAULT: ${webclient.tempdir}/webclient-jobs/).webclient.plandump
: The directory into which the web server will dump
temporary JSON files describing the execution plans
(DEFAULT: ${webclient.tempdir}/webclient-plans/).The parameters define the behavior of tasks that create result files.
fs.overwrite-files
: Specifies whether file output writers should overwrite
existing files by default. Set to true to overwrite by default, false otherwise.
(DEFAULT: false)fs.output.always-create-directory
: File writers running with a parallelism
larger than one create a directory for the output file path and put the different
result files (one per parallel writer task) into that directory. If this option
is set to true, writers with a parallelism of 1 will also create a directory
and place a single result file into it. If the option is set to false, the
writer will directly create the file directly at the output path, without
creating a containing directory. (DEFAULT: false)compiler.delimited-informat.max-line-samples
: The maximum number of line
samples taken by the compiler for delimited inputs. The samples are used to
estimate the number of records. This value can be overridden for a specific
input with the input format's parameters (DEFAULT: 10).compiler.delimited-informat.min-line-samples
: The minimum number of line
samples taken by the compiler for delimited inputs. The samples are used to
estimate the number of records. This value can be overridden for a specific
input with the input format's parameters (DEFAULT: 2).compiler.delimited-informat.max-sample-len
: The maximal length of a line
sample that the compiler takes for delimited inputs. If the length of a single
sample exceeds this value (possible because of misconfiguration of the parser),
the sampling aborts. This value can be overridden for a specific input with the
input format's parameters (DEFAULT: 2097152 (= 2 MiBytes)).


Like traditional data, XML documents can be added to a database table, altered, removed, and retrieved using SQL Data Manipulation Language statements (INSERT, UPDATE, DELETE, and SELECT statements). Typically, XML documents (as defined in the XML 1.0 specification) are manipulated by application programs; when performing DML operations from an application program, IBM recommends that XML data be manipulated through host variables, rather than literals, so DB2 can use the host variable data type to determine some of the encoding information needed for processing. And although you can manipulate XML data using XML, binary, or character types, IBM recommends that you use XML or binary types to avoid code page conversion issues.
XML data used in an application is often in a serialized string format; when this data is inserted into an XML column or when data in an XML column is updated, it must be converted to its XML hierarchical format. If the application data type used is an XML data type, DB2 performs this operation implicitly. However, if the application data type is a character or binary data type, the XMLPARSE() function must be used to convert the data explicitly from its serialized string format to the XML hierarchical format during insert and update operations. A simple INSERT statement that uses the XMLPARSE() function to insert a string value into an XML column named CUSTINFO in a table named CUSTOMERS might look something like this:
INSERT INTO customers (custinfo) VALUES (XMLPARSE(DOCUMENT '<name>John Doe</name>' PRESERVE WHITESPACE))
When the Command Line Processor is used to manipulate XML documents stored in XML columns, string data can be directly assigned to XML columns without an explicit call to the XMLPARSE() function if insert, update, and delete operationsare performed. For example, let's say you want to add a record containing XML data to a table named CUSTOMER that has the following characteristics:
|
Column Name |
Data Type |
|---|---|
|
CUSTID |
INTEGER |
|
INFO |
XML |
You could do so by executing an INSERT statement from the Command Line Processor that looks something like this:
INSERT INTO customer VALUES (1000, '<customerinfo xmlns="http://custrecord.dat" custid="1000"> <name>John Doe</name> <addr country="United States"> <street>25 East Creek Drive</street> <city>Raleigh</city> <state-prov>North Carolina</state-prov> <zip-pcode>27603</zip-pcode> </addr> <phone type="work">919-555-1212</phone><EMAIL_ADDRESS></customerinfo>')
And if you wanted to update the XML data portion of this record from the Command Line Processor, you could do so by executing an UPDATE statement that looks something like this:
UPDATE customer SET custinfo = '<customerinfo xmlns="http://custrecord.dat" custid="1000"> <name>Jane Doe</name> <addr country="Canada"> <street>25 East Creek Drive</street> <city>Raleigh</city> <state-prov>North Carolina</state-prov> <zip-pcode>27603</zip-pcode> </addr> <phone type="work">919-555-1212</phone><EMAIL_ADDRESS></customerinfo>' WHERE XMLEXISTS ('declare default element namespace"http://custrecord.dat"; $info/customerinfo[name/text()="John Doe"]' PASSING custinfo AS "info")
Finally, if you wanted to delete the record from the CUSTOMER table, you could do so by executing a DELETE statement from the Command Line Processor that looks something like this:
DELETE FROM customer WHERE XMLEXISTS ('declare default element namespace "http://custrecord.dat"; $info/customerinfo[name/text()=" John Doe"]' PASSING custinfo AS "info")
So how do you retrieve XML data once it has been stored in a table? With DB2 9, XML data can be retrieved using an SQL query or one of the SQL/XML query functions available. When querying XML data using SQL, you can only retrieve data at the column level-in other words, an entire XML document must be retrieved. It is not possible to return fragments of a document using SQL; to query within XML documents, you need to use XQuery.
XQuery is a functional programming language that was designed by the World Wide Web Consortium (W3C) to meet specific requirements for querying XML data. Unlike relational data, which is predictable and has a regular structure, XML data is often unpredictable, highly variable, sparse, and self-describing. Because the structure of XML data is unpredictable, the queries that are performed on XML data often differ from typical relational queries. For example, you might need to create XML queries that perform the following operations:
Search XML data for objects that are at unknown levels of the hierarchy.
Perform structural transformations on the data (for example, you might want to invert a hierarchy).
Return results that have mixed types.
In XQuery, expressions are the main building blocks of a query. Expressions can be nested and form the body of a query. A query can also have a prolog that contains a series of declarations that define the processing environment for the query. Thus, if you wanted to retrieve customer names for all customers who reside in North Carolina from XML documents stored in the CUSTINFO column of a table named CUSTOMER (assuming this table has been populated with the INSERT statement we looked at earlier), you could do so by executing an XQuery expression that looks something like this:
XQUERY declare default element namespace "http://custrecord.dat"; FOR $info IN db2-fn:xmlcolumn('CUSTOMER.CUSTINFO')/customerinfo WHERE $info/addr/state-prov=" North Carolina" RETURN $info/name
When this XQuery expression is executed from the Command Line Processor, it should return information that looks like this (again, assuming the CUSTOMER table has been populated with the INSERT statement we looked at earlier):
1 ---------------------------------------------------- <name xmlns="http://custrecord.dat">John Doe</name>
If you wanted to remove the XML tags and just return the customer name, you could do so by executing an XQuery expression that looks like this instead:
XQUERY declare default element namespace "http://custrecord.dat"; for $info in db2-fn:xmlcolumn('CUSTOMER.CUSTINFO')/customerinfo where $info/addr/state-prov=" North Carolina" return $info/name/text()
Now, when the XQuery expression is executed from the Command Line Processor, it should return information that looks like this:
1 -------- John Doe
As mentioned earlier, XQuery expressions can be invoked from SQL using any of the following SQL/XML functions or predicates:
XMLQUERY(). XMLQUERY() is an SQL scalar function that enables you to execute an XQuery expression from within an SQL context. XMLQUERY() returns an XML value, which is an XML sequence. This sequence can be empty or it can contain one or more items. You can also pass variables to the XQuery expression specified in XMLQUERY().
XMLTABLE(). XMLTABLE() is an SQL table function that returns a table from the evaluation of XQuery expressions; XQuery expressions normally return values as a sequence, however, XMLTABLE() allows you to execute an XQuery expression and return values as a table instead. The table that is returned can contain columns of any SQL data type, including XML. The structure of the resulting table is defined by the COLUMNS clause of XMLTABLE().
XMLEXISTS. The XMLEXISTS predicate determines whether an XQuery expression returns a sequence of one or more items. If the XQuery expression specified in this predicate returns an empty sequence, XMLEXISTS returns FALSE; otherwise, TRUE is returned. The XMLEXISTS predicate can be used in the WHERE clauses of UPDATE, DELETE, and SELECT statements. This usage means that values from stored XML documents can be used to restrict the set of rows that a DML statement operates on.
By executing XQuery expressions from within the SQL context, you can:
Operate on parts of stored XML documents instead of entire XML documents (only XQuery can query within an XML document; SQL alone queries at the whole document level)
Enable XML data to participate in SQL queries
Operate on both relational and XML data
Apply further SQL processing to the returned XML values (for example, ordering results with the ORDER BY clause of a subselect)
Thus, suppose you wanted to retrieve customer IDs and customer names a table named CUSTOMER that has the following characteristics:
|
Column Name |
Data Type |
|---|---|
|
CUSTID |
INTEGER |
|
INFO |
XML |
You could do so (assuming this table has been populated with the INSERT statement we looked at earlier) by executing a SELECT statement from the Command Line Processor that looks something like this:
SELECT custid, XMLQUERY ('declare default element namespace "http:// custrecord.dat"; $d/customerinfo/name' PASSING custinfo AS "d") AS address FROM customer;
And when this query is executed, it should return information that looks something like this:
CUSTID ADDRESS ------ --------------------- 1000 <name xmlns="http://custrecord.dat">John Doe</name>


Have you seen this essay on churchofjesuschrist.org
https://www.churchofjesuschrist.org/study/manual/gospel-topics-essays/translation-and-historicity-of-the-book-of-abraham
I thought the Book Of Abraham was an actual direct translation of what was on the papyri. Apparantly it is a part of the Book of the Dead. Why does the translation not match the actual text?
That can't be right can it?