Here is some content without 'tags' so that we can see what happens.
Even though I put spaces in my code
the browser still doesn't know that I want seperate paragraphs, because I
have not instructed it to do so.
Tags Are Important to declare to the browser what we want.
h1 Headline
h2 Headline
h3 Headline
h4 Headline
h5 Headline
h6 Headline
paragraph element
Bold and Italics
emphasis / italicized (semantic)
italicized (non-semantic, should only be used as a last resort)
bold (sematic)
bold (non-semantic, should only be used as a last resort)
My name is Ryan Camp
This is the third paragraph of the article.
Lists
Unordered List
unordered list item 1
unordered list item 2
unordered list item 3
Ordered List
ordered list item 1
ordered list item 2
ordered list item 3
definition list / description list
Term 1
A description / definition of Term 1. When I see this I think of a dictionary
Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
Term 2
A description / definition of Term 2. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
Term 3
A description / definition of Term 3. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
Quotes
Block quotes
Anyone who has lost track of time when using a computer knows the propensity to dream, the urge to make dreams come true and the tendency to miss lunch.
- Tim Berners-Lee
Inline quotes using 'q' element
The man who's brainchild is the World Wide Web, Tim Berners-Lee, once said Anyone who has lost track of time when using
a computer knows the propensity to dream, the urge to make dreams come true and the tendency to miss lunch.
Inline vs Block-Level elements
Inline
List of a couple of inline elements
<q>

Inline elements will keep the content inline and not create a newline
Block-Level
list of a couple block-level elements

Block level elements will create a new line and occupy its own block for it's content.
You can use CSS to switch any elements layout from 'inline' to 'block' and visa versa.
Dates and Times
My birthday is at
If your looking at the code we used a 'datetime' attribute to tell the computer what date my birthday is on.
An attribute goes inside the opening tag of an element and signifies some attribute of that element.
There are many formats of content that can go into datetime covering:
date
datetime
time
duration
For a list of some of these formats please visit:w3schools
code, pre, and br

if you want a code snippet to stand out you can use the 'code' element.
10.times { puts 'hello world' } This is a code snippet in ruby that
would disply 'hello world' to the terminal 10 times.

the line break element will break lines inside a paragraph element
if
i
want
each
thing
seperated it won't do it automatically.
Instead
I
will
need
to
break
each
line

the pre tag allows you to control when the browser pays attention to the spacing
and tabing for an entire group of text
I want
this todo
What i want
I want
this todo
What i want
With
 and  put together I can now represent
multi line code snippets. Here is the same snippet with a do block.
10.times do
puts 'hello world'
end
There you have it...


All FAA Part 135 aviation services provided by Metro Aviation, Inc., which maintains exclusive operational control over all aircraft.
Join us to kick off the greatest month in racing at Rev, presented by Fifth Third Bank, and hosted at the Indianapolis Motor Speedway.
Guests will dine on driver-inspired cuisine prepared by Indy’s top chefs and enjoy a variety of music and entertainment while mingling with IndyCar drivers and racing legends. Experience a behind-the-scenes look at the Yard of Bricks, the acclaimed Borg-Warner trophy and Victory Podium and enjoy photo opportunities with Indianapolis’s finest vintage cars.
Funds raised support Indiana University Health statewide trauma and critical care programs. These funds provide care for drivers and patrons at the Indiana University Health Emergency Medical Center located at Indianapolis Motor Speedway and for patients across Indiana in need of critical care.
VIP Elite ticket holders will enjoy early access to the event with complimentary valet parking and a private reception in the world famous Pagoda that will feature hand crafted cocktails and unique culinary specialties. VIP Elite guests will also experience thrilling speeds over 100+ mph around the Indianapolis Motor Speedway during early access to the track. Limited number of tickets available.
To become a sponsor, contact Carol Howard
Contact Carol Howard
Benefits:
Benefits:
Benefits:
Benefits:
EVENT LOUNGE SPONSOR
HOT LAP SPONSOR
MAIN STAGE SPONSOR
PROMOTIONAL ITEM SPONSOR SOLD OUT
RED CARPET SPONSOR SOLD OUT
CUSTOM PHOTO BOOTH SPONSOR
Benefits:
COURTYARD CHANDELIER BAR SPONSOR
VALET SPONSOR
EVENT ENTRANCE SPONSOR SOLD OUT
PHOTO BOOTH SPONSOR SOLD OUT
STAGE SPONSOR
Benefits:
Benefits:
Benefits:
Contact Carol Howard to begin customizing your company’s sponsorship!
Interested in volunteering at Rev?
Email<EMAIL_ADDRESS>General Event Information, Corporate Sponsorships & Ticket Packages
Contact Carol Howard
IU Health Methodist Hospital and the Indianapolis Motor Speedway have a relationship dating to the racetrack’s earliest days and, like the Indianapolis 500, are poised to move forward for years to come. From the hospital’s first motorized ambulance in 1910 to the first liftoff of the LifeLine helicopter in 1970 to today, IU Health Methodist Hospital and IMS have worked together to promote the safety of race car drivers and Hoosiers alike. As we approach the 101st Running of the Greatest Spectacle in Racing, Rev is a celebration of the continued partnership between these influential Hoosier institutions.
Indiana University Health
www.iuhealth.org
Indianapolis Motor Speedway
4790 West 16th Street, Indianapolis, IN 46222
www.indianapolismotorspeedway.com
Verizon IndyCar Series
www.indycar.com
102nd Running of the Indianapolis 500
www.indy500.com


20 Colorized Historical Photos That Open a Window to the Past
When we open ordinary history books, all we see there are boring old black and white photos that seem to have been taken so far away in time from us. This is probably why we always forget that people back then were actually real, with their own joys and sorrows. To make us feel a little bit closer to our ancestors and understand their way of living better, internet users decided to add color to the pictures taken in previous centuries.
Bright Side suggests that you take a quick tour through the past and have a look at these 20 colorized photos we’ve chosen for you.
20. Elizabeth Taylor, 1956
19. Otto von Bismarck, 1894
18. Grace Kelly, 1954
17. A window washer working on the Empire State Building, 1936
16. Marilyn Monroe, 1951
15. Clint Eastwood, 1956
14. Walt Disney with Donald Duck, 1951
13. The Beatles, 1962
12. Young survivors of the Titanic, 1912
11. Ava Gardner, circa 1942
10. Miss America 1924
9. A woman in a polka pot dress, New York, 1946
8. John Lennon and Yoko Ono, 1972
7. A Welsh boy with a lamb, 1936
6. Behind Big Ben, 1859
5. Mark Twain, circa 1883
4. A drinking fountain in Central Park, New York, 1942
3. Mata Hari, 1910
2. Alan Milne with his son Christopher Robin, 1926
1. Elizabeth Taylor in the 1940s
Which of these photos is the most impressive, in your opinion? Do you have your own old photos that you’d like to see colorized? Please share them with us in the comments!
Preview photo credit Frank Worth, Courtesy of Capital Art / Contributor / Getty Images, malakon / reddit


-
We are a New Age App Company ready to coupled with a eco system to establish more business opportunities.
-
Following the foot-step of the most successful business platform "Alibaba" and visonary Mr Jack Ma, VGLOBAL aims to be new rookie in app industry globally.
By 2019, we should be ready to launch our first group of shares to all our shareholders. And its possible cause, by then, we should have already a big pool of consumers and merchants. Not just that, it would also mean having a substantial amount of assets and capbilities.


Hadley Centre refereed Publications (1990)
- Climate Simulation/prediction
- Ocean Modelling
- Climate variability/data
- Physical processes/parametrization
- Miscellaneous
- Long-Range Forecasting
Contact information
Andy Brady |<EMAIL_ADDRESS>| Last updated: 28 September 1995 | © Crown Copyright 1995


"BOND "an obligation of any kind (Numbers 30:2, 4, 12). The word means also oppression or affliction (Psalm 116:16; Phil. 1:7). Christian love is the“bond of perfectness” (Colossians 3:14), and the influences of the Spirit are the “bond of peace” (Ephesians 4:3).
"BOND "an obligation of any kind (Numbers 30:2, 4, 12). The word means also oppression or affliction (Psalm 116:16; Phil. 1:7). Christian love is the“bond of perfectness” (Colossians 3:14), and the influences of the Spirit are the “bond of peace” (Ephesians 4:3).


open class EngineConnectorBuilder : EngineConnectorConfig
EngineConnectorBuilder(type: ConnectorType = ConnectorType.HTTP) |
open var host: String
The network interface this host binds to as an IP address or a hostname. If null or <IP_ADDRESS>, then bind to all interfaces. |
|
open var port: Int
The port this application should be bound to. |
|
open val type: ConnectorType |
open fun toString(): String |
class EngineSSLConnectorBuilder : EngineConnectorBuilder, EngineSSLConnectorConfig |


Make sure this window is full screen. You should see four little popup windows with shifting colors right below this text. If you don't, you may need to allow popups for this site.
Click on the edges of a window and watch the bubbles flow from one window to the others in synchronized fashion!


The Obesity Epidemic
Many people around the world have been suffering the consequnces from high caloric food.
Read More >>>
The S.S. Satire is an informational website that delivers information about today's important issues. Check out our newest article on the obesity epidemic today!
Many people around the world have been suffering the consequnces from high caloric food.
Read More >>>
Reports have stated that standardized tests actually lower student IQ and negatively affect the school enviroment.
Read More >>>
Youtuber Cody's Lab has made videos about mining and forging. Watch as he creates a ring out of precious metals in his video!
 The media is not shocked!
Read more >>>
Policemen and many locals of Seattle, Washington report on seeing a yellow disc-shaped object floating in the sky at night, followed by a strong scent of bananas.
Read more >>>


Curtain / Materials
Celebrity
This exquisite collection of decorative embroidery on translucent sheers evokes glamorous luxury. Richly ornamental patterns, geometrics and vivid leaf motifs in brown and gold on white fabric are augmented by stardust-like sprinklings of sequins, shiny stones, silver highlights and zari. Ideal draping for lavish, jubilant interiors that exude an exclusive charm.


IRC Bot
The Project
This tutorial walks you through how to make an IRC bot with Twisted. You will be introduced to testing, logging, an overview of how the internet works, as well as event-driven programming, different internet protocols, and making a portable application.
The project’s code is based off of Jessamyn Smith’s IRC bot – the talkbackbot, where if anyone says “That’s what she said” in an IRC channel, the bot replies with a notable quote from a woman (that’s what she really said!).
Intro to IRC
IRC stands for Internet Relay Chat, and is a protocol for live messaging over the internet. Specifically, the IRC protocol is within the application layer (just a simple abstraction of types of protocols). Other examples of internet protocols within the application layer are HTTP, IMAP, SMTP, POP, SSH, among many others.
IRC uses TCP, Transmission Control Protocol (another protocol!). TCP is within the transport layer, as opposed to application later. Another popular transportation layer protocol is UDP.
If you have heard of IRC before, you may think it’s an antiquated means of communication. Created in 1988, it remains one of the most popular protocols for instant messaging within many aspects of the tech community.
There are many IRC networks, the most popular in terms of users at peak hours are freenode, IRCNet, and QuakeNet.
IRCHelp is a great resource for learning IRC. PyLadies also has a great introduction on how to setup and use IRC. Of the Python community, some frequent channels visited are #python, #django, #python-dev, #twisted, and of course, #pyladies, all on Freenode.
For more information, Jessica McKellar gave a great talk at PyCon 2013 about How the Internet Works, with her slides here.
Intro to Twisted
According to Twisted’s website, Twisted is an “event-driven networking engine written in Python”. OK…what does that mean?
Event-driven programming
Event-driven programming simply means that the flow of the program is determined by events. Events may be a click of the mouse, a message from another program, a response from a server, etc. The primary activity is a reaction to receiving a certain event(s).
Glyph, the author of Twisted, gave a great talk on event-driven architecture at PyCon 2013. He describes how to approach thinking about events and callbacks the right way. Essentially, an event is just a function call that you asked for. Glyph’s slides encapsulates it well:
- When a user clicks a button, then call this function.
- When a user answers this question, call this function.
- When a network answers a question, call this function.
- When a network sends me data, call this function.
Programming Paradigms
Some paradigms include object-oriented, imperative, and functional, all of which can use event-driven programming. If you are curious about how to employ different programming paradigms for different problems, check out How to Design Programs.
Intro to Logging
Logging is quite important for applications. Transferring money, the black box on an airplane, cell phone bills - they all log actions to be referenced and checked later. As such, logging is quite important for developing, debugging, maintaining, and running systems and applications.
There is the logging
module in Python’s standard library, and a great how-to write up for the logging
module.
But Twisted has its own logging module, log
. The initial reason that Twisted doesn’t use the logging
module from the Python standard library is that the Twisted logging module predates the stdlib one. There are many reasons that Twisted hasn’t moved to the stdlib logging module which you can read here.
You may start out simply adding print
statements to your application, but this isn’t ideal for anything beyond a very basic script. print
does not allow you to set up different importance levels, e.g. DEBUG
, INFO
, WARN
, ERROR
, CRITICAL
, or FAIL
; it's an all or nothing with print
.
Identifying which level of importance to log a message at can get some getting use to. Use the debug
level for granular information, such as printing out variables that change within a for-loop:
1 2 3 4 | def some_awesome_function(items):
for i, item in enumerate(items):
# do some complex computation/iteration
logger.debug('%s iteration, item=%s', i, item)
|
Use info
for routines and such, like starting or connecting to a server:
1 2 3 4 | def start_IMAP_service():
logger.info('Starting IMAP service at port %s ...', port)
service.start()
logger.info('IMAP Service is started')
|
Use warn
for important events happen, but no error has occurred, like a password that was incorrectly inputted. Finally, use error
for when an exception is thrown, user isn’t found in the database, connectivity issue, etc. As the admin of an application with a logging mechanism, you would setup your desired level of logs in some configuration for instance, if you only want to see errors, you would ideally set a configuration value to something like debug_level=error
.
These above examples use Python’s logging
module. For our tutorial, we’ll use Twisted’s log
module, which has slightly different syntax when passing in log levels.
Intro to Testing
Very similar to logging, testing is also quite important for your code base. Writing tests in parallel to writing code is considered a good habit. Submitting features and patches to projects without tests can be a big faux-pas.
There are different types of testing, and with this tutorial we will focus on unit testing.
A unit test just focuses on one tiny bit of functionality to prove correct behavior based on inputs, and should be independent of other unit tests.
An example of a unit test using Pythons unittest
module, taken from python-guide.org:
1 2 3 4 5 6 7 8 | import unittest
def fun(x):
return x + 1
class MyTest(unittest.TestCase):
def test(self):
self.assertEqual(fun(3), 4)
|
The main part here is assertEqual(fun(3), 4)
, where we feed fun
the number 3
. The test will pass if the return value of fun(3) == 4
, else it will fail.
We will be using Twisted’s own unit testing framework that is built upon Python’s unittest
module with the added ability to test event-driven code.
Approaching the tutorial
We will first knock out the simple items by addressing the global variables/settings for our IRC bot. We’ll then build a quick function to select quotes (a list of quotes can be found in the GitHub repo).
Then we’ll approach the making of the bot module by first creating a class to create bots (a factory class), and then another class to define the behavior of the bot. We will also write a quick plugin to easily create and start up our bot from the command line.
Lastly, we will write tests for the expected behavior of our talkback bot application.
All set? Let’s begin with our setup!


This page contains basic models, which are applied to the modeling of the learning process and to decision-making in the classroom.
Simple two-dimensional optimization
The focus in this section is on the decision making of the learner. It has been adapted from Fourer et al. (1990).
A learner must decide how to allocate next week’s studying time. She has to assimilate two types of competencies: knowledge and application. The first skill (knowledge) requires that the learner knows the terminology, theory, and techniques in a given subject. The second skill (application) requires the learner to apply the knowledge to particular problems or issues. Learners have to recognise which terms or concepts apply to a particular problem or situation. For achieving these skills, the learner must memorize the knowledge and make exercises, in which the knowledge is applied.
The learner is constrained to a limited number of pages that she can study in an hour. This means that the representative learner cannot memorize more than 20 pages per hour. Similarly, she cannot study more than 14 pages of exerises per hour.
The two skills will affect the final mark in different ways. We assume that the profitability obtained per page memorized and applied is respectively given by: 25 marks and 30 marks.
The following weekly studying amounts are the most that can be achieved in the light of the time constraints:
-
Memorization of knowledge: 600 pages
-
Exercises: 400 pages
The question that the learner faces is: If 40 hours of studying time are available this week, how many pages of memorization and how many pages of exercises should be studied to bring the greatest total profit or outcome in terms of final marks?
While we are given numeric values for number of pages that can be studied and marks that can be achieved per page, the pages of memorization and of exercises to be produced are as yet unknown. These quantities are the decision variables whose values we must determine so as to maximize profits.
The purpose of the linear program is to specify the profits and production limitations as explicit formulas involving the variables, so that the desired values of the variables can be determined systematically.
We will write \(X_M\) for the number of pages of memorization to be studied, and \(X_E\) for pages of exercises. The total hours to study all these pages is then given by:
\begin{equation} (\mbox{hours to study one page of memorization}) \times X_M + (\mbox{hours to study one page of exercises}) \times X_E \nonumber \end{equation}
This number cannot exceed the 40 hours available. Since hours per page is the reciprocal of the pages per hour given above, we have a constraint on the variables:
\begin{equation} \frac{1}{20} X_M + \frac{1}{14} X_E \leq 40 \nonumber \end{equation}
Pages that can be studied per week are also limited:
\begin{equation} 0 \leq X_M \leq 600 \nonumber \\ 0 \leq X_E \leq 400 \nonumber \end{equation}
In this problem, the upper limits were specified, but the lower limits were assumed. It is obvious that a negative number of pages that can be studied would be meaningless. For computational reasons, a lower limit has to be defined.
By analogy with the formula for total hours, the total outcome in marks must be:
\begin{equation}
(\mbox{outcome per page of memorization}) \times X_M
(\mbox{outcome per page of exercises}) \times X_E \nonumber
\end{equation}
That is, our objective is to maximize \(25 X_M + 30 X_E\). Putting this all together, we have the following linear program:
\begin{equation} \max 25 X_M + 30 X_E \nonumber \\ \hspace{1cm}\mbox{s.t.} \frac{1}{20} X_M + \frac{1}{14} X_E \leq 40 \nonumber \\ \hspace{2cm} 0 \leq X_M \leq 600 \nonumber \\ \hspace{2cm} 0 \leq X_E \leq 400 \nonumber \end{equation}
The horizontal line represents the studying limit on memorization, the vertical on exercises. The diagonal line is the constraint on hours, each point on that line represents a combination of memorization and exercises that requires exactly 40 hours of studying time, and any point downward and to the left requires less than 40 hours.
The shaded region bounded by the axes and these three lines corresponds exactly to the feasible solutions - those that satisfy all three constraints. Among all the feasible solutions represented in this region, we seek the one that maximizes the profit.
For this problem, a line of slope \(-\frac{25}{30}\) represents combinations that produce the same profit, for example, the line from (0, 450) to (540, 0) represents combinations that yield ??? points of outcome.
Different total marks of outcome give different but parallel lines in the figure, with higher total marks of outcome giving lines that are higher and further to the right.
Solving the Model using AMPL language in NEOS
The following block gives the Ampl code for the previous problem, which
should be saved in a separate file (study0.mod
)
var XM;
var XE;
maximize Profit: 25 * XM + 30 * XE;
subject to Time: (1/20) * XM + (1/14) * XE <= 40;
subject to M_limit: 0 <= XM <= 600;
subject to E_limit: 0 <= XE <= 400;
Create another file that contains the following instructions (study0.run
)
solve;
display XM, XE;
These two files can be uploaded to the NEOS server. Note: You must register to the Website (it won’t cost you anything).
This code can be written in a more general form as follows:
The file that contains the model will look like this (study.mod
):
set P;
param a {j in P};
param b;
param c {j in P};
param u {j in P};
var X {j in P};
maximize Total_Profit: sum {j in P} c[j] * X[j];
subject to Time: sum {j in P} ( 1/a[j] ) * X[j] <= b;
subject to Limit {j in P}: 0 <= X[j] <= u[j];
The parameter values can be stored in a separate file (study.dat
):
set P := memorization, exercise;
param: a c u :=
memorization 20 25 600
exercise 14 30 400;
param b := 40;
And finally the instructions for the optimizer can be collected in a separate
file (study.run
):
solve;
display X;
Running this general specification can be done by uploading the three files on the NEOS server.
This will give the following message from NEOS:
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Checking ampl.mod for cplex_options...
Checking ampl.com for cplex_options...
Executing AMPL.
processing data.
processing commands.
Executing on prod-exec-5.neos-server.org
Presolve eliminates 2 constraints.
Adjusted problem:
2 variables, all linear
1 constraint, all linear; 2 nonzeros
1 inequality constraint
1 linear objective; 2 nonzeros.
CPLEX <IP_ADDRESS>: threads=4
CPLEX <IP_ADDRESS>: optimal solution; objective 19200
1 dual simplex iterations (0 in phase I)
X [*] :=
exercise 140
memorization 600
;
Solving the Model using JuMP in Julia
The same problem can be solved using the JuMP package in Julia (Julia code):
# Launch the script from Tearminal: include("scriptname.jl")
# Source: https://hackernoon.com/linear-optimization-using-julia-40311c0ad98c
using JuMP
using GLPK
# Create model
lpModel = Model(GLPK.Optimizer)
@variable(lpModel, XM <= 600)
@variable(lpModel, XE <= 400)
# Create an objective
@objective(lpModel, Max, 25*XM + 30*XE)
# Create constraints
@constraint(lpModel, (1/20)*XM+(1/14)*XE <= 40)
# Solve and read the answer
optimize!(lpModel)
println("Value of objective function : $(objective_value(lpModel))" )
println("Value of XM (memorization) : $(getvalue(XM))")
println("Value of XE (exercise) : $(getvalue(XE))")
The previous code will give the following results:
Value of objective function : 19200.0
Value of XM (memorization) : 600.0
Value of XE (exercise) : 140.0
Simple Model: Inference
The focus in this section is on the decision making of the teacher. It has been adapted from Gelman et al. (1995) and Lunn et al. (2013).
When designing a lesson, the teacher has to decide on the ratio knowledge to application, \(kn/app\), treated in a lesson. The learner’s responses are typically characterized by a dichotomous outcome: for example, "has acquired the competency" or "failed to acquire the competency".
Qn experimental lesson of this kind gives rise to data of the form \begin{equation} (x_i, n_i, y_i); i=1, \ldots, k, \end{equation} where \(x_i\) represents the i\(th\) of \(k\) lessons with different ratios \(kn/app\). \(n_i\) represents the number of learners present in the lesson, of which \(y_i\) subsequently respond with positive outcome.
An example is shown in the table below: a sequence of four lessons with ten learners in each lesson has been tried.
| Ratio, \(x_i\) (log (\(kn(min)/app(min)\)) ) |
Number of learners in the lesson |
Number of learners who failed to acquire the competency |
|---|---|---|
-0.92 |
10 |
2 |
-0.69 |
10 |
4 |
-0.36 |
10 |
6 |
0.40 |
10 |
8 |
If we assume that 15 min of a 50 min lesson is dedicated to presenting theory and knowledge, and 35 min is dedicated to applying that knowledge, we will have a ratio \(15/35\) or \(3/7\), which would be: log(\(3/7\))=-0.85\(\equiv x_i\).
We will model the outcomes of the ten learners within each group \(i\) as exchangeable, and it seems reasonable (if we assume that these are different classes) to model them as independent with equal probabilities, which implies that the data points \(y_i\) are binomially distributed:
\begin{equation} y_i | \theta_i \sim Bin(n_i, \theta_i) \nonumber \end{equation}
where \(\theta_i\) is the probability of failing the targeted competency for the learner exposed to a lesson of type \(x_i\).
It should be noted that a situation in which independence and the binomial model would not be appropriate is if the learners are solving problems in groups - meaning with knowledge transfer from one learner to the other.
For the case described above, we assume that there is no groupwork and that the four classes are different classes with different (not overlapping) learners, given the parameters \(\theta_1, \ldots, \theta_4\).
The simplest analysis would treat the four parameters \(\theta_i\) as exchangeable in their prior distribution, perhaps using a noninformative density such as \(p(\theta_1, \ldots, \theta_4) \propto 1\), in which case the parameters \(\theta_i\) would have independent beta posterior distributions. The exchangeable prior model for the \(\theta_i\) parameters has a serious flaw, however; we know the ratio level \(x_i\) for each group \(i\), and one would expect the probability of failing to vary systematically as a function of the ratio.
The simplest model of ratio-response relation — that is, the realtion of :\(\theta_i\) to \(x_i\) — is linear: \(\theta_i = \alpha + \beta x_i\).
Unfortunately, this model has the flaw that at low or high values of the ratio, \(x_i\) approaches \(\pm \infty\), whereas \(\theta_i\), being a probability, must be constrained to lie between 0 and 1.
The standard solution is to use a transformation of the \(\theta_i\)'s, such as a logistic, in the ratio-response relation:
\begin{equation} logit(\theta_i) = \alpha + \beta x_i \end{equation}
where \(logit(\theta_i) = log( \theta_i / (1+\theta_i) )\). This is called a logistic regression model.
One approach to estimate th eparameters of this model is to use the BUGS software.
We fit the following logistic regression model:
\begin{equation} y_i \sim Binomial(p_i, n_i) \\ logit(p_i) = \alpha + \beta(x_i - \bar{x}) \end{equation}
with vague \(Normal(0, 100^2)\) priors for \(\alpha\) and \(\beta\).
Note that, the covariate (ratio) is centred, by subtracting \(\bar{x}=N^{-1} \sum x_i\). This is because serious MCMC convergence issues arise when the \(x_i\)'s are used directly, due to very high posterior correlation between \(\alpha\) and \(\beta\).
Centering essentially relocates the \(y\)-axis to \(x=\bar{x}\), which, in this case, vastly reduces the dependence of the intercept \(\alpha\) on \(\beta\).
The likelihood is specified as follows:
for (i in 1:4) {
y[i] ~ dbin( p[i], n[i] )
logit(p[i]) <- alpha + beta * ( x[i] - mean(x[]) )
}
If we want to assess the model fit visually then we will need to either transform the data onto the same scale as the linear model or transform the model fit onto the same scale as the observations.
We insert the following code in the loop above so that we can examine both:
phat[i] <- y[i]/n[i]
yhat[i] <- n[i]*p[i]
Under the model described earlier, we can write the sampling distribution, or likelihood, for each group \(i\) in terms of the parameters \(\alpha\) and \(\beta\) as
\begin{equation} p(y_i \mid \alpha, \beta, n_i, x_i) \propto [logit^{-1}(\alpha + \beta x_i)]^{y_i} [1-logit^{-1}(\alpha + \beta x_i)]^{n_i-y_i} \end{equation}
The model (BUGS code) is characterized by the parameters \(\alpha\) and \(\beta\), whose joint posterior distribution is
\begin{equation} p(\alpha, \beta \mid y, n, x) \propto p(\alpha, \beta \mid n, x) p(y \mid \alpha, \beta, n, x) \\ \propto p(\alpha, \beta) \prod_{i=1}^{k} p(y_i \mid \alpha, \beta, n_i, x_i) \end{equation}
We consider the sample sizes \(n_i\) and ratio levels \(x_i\) as fixed for this analysis and suppress the conditioning on \((n, x)\) in subsequent notation.
We present an analysis based on a prior distribution for \((\alpha, \beta)\) that is independent and locally uniform in two parameters; that is \(p(\alpha, \beta) \propto 1\). In practice, we might use a uniform prior distribution if we really have no prior knowledge about the parameters, or if we want to present a simple analysis of this experiment alone.
If the analysis using noninformative prior distribution is insufficiently precise, we may consider using other sources of substantive information (for example, from other classroom experiments) to construct an informative prior distribution.
Another tool for solving this problem is the Stan software (cmdstan
).
The previous model can be implemented as follows:
data {
int<lower=0> N;
int<lower=0> n[N];
int<lower=0> r[N];
vector[N] x;
}
transformed data {
vector[N] centered_x;
real mean_x;
mean_x <- mean(x);
centered_x <- x - mean_x;
}
parameters {
real alpha_star;
real beta;
}
transformed parameters {
vector[N] m;
m <- alpha_star + beta * centered_x;
}
model {
alpha_star ~ normal(0.0, 1.0E4);
beta ~ normal(0.0, 1.0E4);
r ~ binomial_logit(n, m);
}
generated quantities {
real alpha;
real p[N];
real llike[N];
real rhat[N];
for (i in 1:N) {
p[i] <- inv_logit(m[i]);
llike[i] <- r[i]*log(p[i]) + (n[i]-r[i])*log(1-p[i]);
rhat[i] <- p[i]*n[i]; // fitted values
}
alpha <- alpha_star - beta*mean_x;
}
The initial values are given in a separate file:
alpha_star <- 0
beta <- 0
Finally, the data is also stored in a separate file:
x <- c(-0.92, -0.69, -0.36, 0.40)
n <- c(10, 10, 10, 10)
r <- c(2, 4, 6, 8)
N <- 4
Running the previous code in the Terminal can be done as follows:
STAN_HOME = ../../../../..
BOOSTPATH = pass:[\((shell find \)](STAN_HOME)/lib -path '*lib/boost_*' -regex '.*lib\/boost_[^/]*')
EIGENPATH = pass:[\((shell find \)](STAN_HOME)/lib -path '*lib/eigen_*' -regex '.*lib\/eigen_[^/]*')
CPPFLAGS = -I pass:[\((BOOSTPATH) -I\)](EIGENPATH) -I $(STAN_HOME)/src
LIBFLAGS = -L$(STAN_HOME)/bin -lstan
$(STAN_HOME)/bin/stanc --name=study study.stan
g++ -O0 -DNDEBUG pass:[\((CPPFLAGS) study_logit.cpp -o study \)](LIBFLAGS)
./study data=study-stan-logit-data.R sample
Note: On my Mac, STAN_HOME is located at /Users/…/opt/anaconda3/envs/stan-env/ . And the stanc
binary is located at /Users/…/opt/anaconda3/envs/stan-env/bin/cmdstan/bin/ .
For more details on checking the Stan compiler.
To Do
-
The data and initial values in the Stan code are in R → Convert into Julia! -
Explain how to read data when running cmdstan in the Terminal -
How to read the initial values when running cmdstan?
References
Fourer, Robert, David M. Gay, and Brian W. Kernighan. "A modeling language for mathematical programming." Management Science 36, no. 5 (1990): 519-554.
Gelman, Andrew, John B. Carlin, Hal S. Stern, and Donald B. Rubin. Bayesian data analysis. Chapman and Hall/CRC, 1995.
Hongwei, Jin. A Tutorial of AMPL for Linear Programming
Lunn, David, Christopher Jackson, Nicky Best, Andrew Thomas, and David Spiegelhalter. "The BUGS book." A Practical Introduction to Bayesian Analysis, Chapman Hall, London (2013).
Stan Development Team. YEAR. Stan Modeling Language Users Guide and Reference Manual, VERSION. https://mc-stan.org
Stan Development Team. stan-dev at Github
StanJulia. CmdStan Installation
Wright, Stephen, and Jorge Nocedal. " Numerical optimization." Springer Science 35, no. 67-68 (1999): 7.


1
Welcome to SeenIt
Share interesting links and discuss great content. It's what's happening now.
Sign in or sign up in a second.
Submit Link
Please, fill out the form. A thumbnail image is not required.
Edit Link
Please, fill out the form. A thumbnail image/description is not required.


This page contains traditional unit tests for quail itself. It uses the
__testQuail
variable exposed out of the closure in
test/quail-testing.jquery.js
.
| Column A | Column B | Column C | Column D |
|---|---|---|---|
| Column A | Column B | Column C | Column D |
| Column A | Column B | Column C | Column D |
| Column A | Column B | Column C | Column D |
| Column A | Column B | Column C | Column D |
| Column A | Column B | Column C | Column D |
| Column A | Column B | Column C | Column D |
| Column A | Column B | Column C | Column D |
| Column A | Column B | Column C | ||
| Column A | Column B | Column C | ||
| Column A | Column B | Column C | ||
| Column A | Column B | Column C |
| This is a sidebar! | This is the content of my page! |
This is the content.
This is an element within the target.
This is readable!
First paragraph
Second paragraph


Introduction to my JacaScript page
This page is made as a project for FreeCodeCamp. I will not duplicate the project example but list a couple of different things that are used when coding with JavaScript.
My reference will be the training that is provided by JavaScript.info.
Since I’m still a student you should not accept that my explanations are 100% correct and I advise that you do some research yourself for more detail.
You can make use of FreeCodeCamp and JavaScript.info as a reliable source of information.
-
Topics to be covered
- Data types
- Operators
- Math operators
- Assignment operators and Comparison operators
We will be looking at an example of code in every section on at least one item explained.
This is an example of how the code will look like. Example from JavaScript.info
let name = "John";
embed a variable
alert( `Hello, ${name}!` ); // Hello, John!
embed an expression
alert( `the result is ${1 + 2}` ); // the result is 3
The expression inside ${…} is evaluated and the result becomes a part of the string.
We can put anything there: a variable like name or an arithmetical expression like 1 + 2 or something more complex.
Please note that this can only be done in backticks. Other quotes do not allow such embedding!


{{newUserHeader}}
{{savecode}}
You will be redirected shortly
Regression Analysis by Team 1d4
Are you a new user or an existing user?
New User
Existing User
Please input your save code here
Submit
Return
{{existingUserStatus}}


[gecode-users] Visual Studio 2008
Christian Schulte
cschulte at kth.se
Tue Jul 22 21:32:10 CEST 2008
Yes, it is indeed supported: I develop Gecode using MSVC 2008 myself.
You sure the compilation went okay?
Christian
--
Christian Schulte, www.ict.kth.se/~cschulte/
From: users-bounces at gecode.org [mailto:users-bounces at gecode.org] On Behalf
Of Nick Hindle
Sent: Monday, July 21, 2008 7:46 PM
To: users at gecode.org
Subject: [gecode-users] Visual Studio 2008
Hi,
I've downloaded Gecode 2.1.1 and built some of the examples using Visual C++
Express 2008, but at runtime I get a memory allocation error in the print
function. It appears this is because none of the IntVar values have been
initialised by the branching process.
Can you confirm if VC++ 2008 supported?
Thanks,
Nick
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ps.uni-sb.de/pipermail/users/attachments/20080722/10c79b62/attachment.htm>
More information about the gecode-users
mailing list


*
Start something new ...
Life is what happens to you while you’re busy making other plans.
–John Lennon
New
File
Open
File
Save
File
Save
File
as
Export
File to
HTML
Export
File to
PDF
Close
Menu
Recent Files:
README.md
notes.md


| CONTENTS |
^H^H^H are ASCII backspace characters. Written printably in email and Usenet messages, they're a tongue-in-cheek way of "erasing" the characters before without actually erasing them. They let you say "I didn't want you to see that" when you actually do.
sed (stream editor) amazes me. Why? It's not just that sed can edit data as it streams through a pipe (like all well-behaved Unix filters (Section 1.5) do). sed can test and branch and substitute and hold and exchange data as it streams through, but so can almost any scripting language. Maybe it's the minimalist in me that loves a tiny program (by today's standards, at least) with just a few operations but operations so well-chosen that they make the tool powerful for its size. Sure, sure, Perl probably can do everything that sed can and do each of those things in twenty different ways. Ah, I've got it: when I'm trying to do anything more than a simple substitution on data streaming by, sed's elegant simplicity almost forces me to strip a problem to its basics, to think of what I really need to do. No functions, no libraries, nothing except beautifully simple functionality.
[As someone who learned Perl regular expressions before I learned sed, I can relate to what Jerry is saying. One of the things I like about the classic Unix toolbox programs like sed is that they really do force you into a sort of Shaker-like elegant simplicity; the best programs, no matter what the language, have a quality like a Shaker chair: pure function, but with a respect for the fact that function doesn't have to be ugly. SJC]
End of sermon. ;-) Even if you aren't into elegance and simplicity, and you just wanna get the job done, what do we cover about sed that might be useful?
In this chapter, we start out with the basics: Section 34.2, Section 34.3, Section 34.4, Section 34.5, Section 34.6, and Section 34.7 show you how to get started, how to test your scripts, and how to structure more advanced scripts. Section 34.8 through Section 34.14 cover regular expressions and complex transformations. Section 34.15 through Section 34.24 deal with advanced topics such as multiline matching and deletions, tests, and exiting a script when you're done.
JP and SJC
If you are already familiar with global edits in other editors like vi or ex, you know most of what you need to know to begin to use sed. There are two things, though, that make it very different:
It doesn't change the file it edits. It is just what its name says: a "stream editor" designed to take a stream of data from standard input (Section 43.1) or a file, transform it, and pass it to standard output (Section 43.1). If you want to edit a file, you have to write a shell wrapper (Section 34.4) to capture standard output and write it back into your original file.
sed commands are implicitly global. In an editor like ex, the command:
s/old/new/
will change "old" to "new" only on the current line unless you use the global command or various addressing symbols to apply it to additional lines. In sed, exactly the opposite is true. A command like the one above will be applied to all lines in a file. Addressing symbols are used to limit the extent of the match. (However, like ex, only the first occurrence of a pattern on a given line will be changed unless the g flag is added to the end of the substitution command.)
If all you want to do is make simple substitutions, you're ready to go. If you want to do more than that, sed has some unique and powerful commands.
This chapter makes no attempt to cover everything there is to know about sed. For the most part, this chapter simply contains advice on working with sed and extended explanations of how to use some of its more difficult commands.
TOR
If you were using sed on the fly, as a stream editor (Section 34.2), you might execute it as simply as this:
% somecommand | sed 's/old/new/' | othercommand
Given filenames, sed will read them instead of standard input:
% sed 's/old/new/' myfile
A simple script can go right on the command line. If you want to execute more than one editing command, you can use the -e option:
% sed -e 's/old/new/' -e '/bad/d' myfile
Or you can use semicolons (;), which are a sed command separator:
% sed 's/old/new/; /bad/d' myfile
Or (especially useful in shell scripts (Section 1.8)) you can use the Bourne shell's ability to understand multiline commands:
sed ' s/old/new/ /bad/d' myfile
Or you can put your commands into a file and tell sed to read that file with the -f option:
% sed -f scriptfile myfile
There's only one other command-line option: -n. sed normally prints every line of its input (except those that have been deleted by the editing script). But there are times when you want only lines that your script has affected or that you explicitly ask for with the p command. In these cases, use -n to suppress the normal output.
TOR
All but the simplest sed scripts are often invoked from a "shell wrapper," a shell script (Section 35.2) that invokes sed and also contains the editing commands that sed executes. A shell wrapper is an easy way to turn what could be a complex command line into a single-word command. The fact that sed is being used might be transparent to users of the command.
Two shell scripts that you should immediately arm yourself with are described here. Both use a shell for loop (Section 35.21) to apply the same edits to any number of files. But the first just shows the changes, so you can make sure that your edits were made correctly. The second writes the edits back into the original file, making them permanent.
Go to http://examples.oreilly.com/upt3 for more information on: checksed
The shell script checksed automates the process of checking the edits that sed makes. It expects to find the script file, sedscr, in the current directory and applies these instructions to the input files named on the command line. The output is shown by a pager program; the default pager is more .
#! /bin/sh script=sedscr for file do echo "********** < = $file > = sed output **********" sed -f $script "$file" | diff "$file" - done | ${PAGER-more}
For example:
$ cat sedscr s/jpeek<EMAIL_ADDRESS>$ checksed home.html new.html ********** < = home.html > = sed output ********** 102c102 < <a<EMAIL_ADDRESS>it or use this form: -- > <a<EMAIL_ADDRESS>it or use this form: 124c124 < Page created by: <a href="mailto:jpeek<EMAIL_ADDRESS>-- > Page created by: <a href="mailto:jpeek<EMAIL_ADDRESS>********** < = new.html > = sed output ********** 22c22 < <a<EMAIL_ADDRESS>comments to me! --- > <a<EMAIL_ADDRESS>comments to me!
If you find that your script did not produce the results you expected, perfect the editing script and run checksed again.
Go to http://examples.oreilly.com/upt3 for more information on: runsed
The shell script runsed was developed to make changes to a file permanently. It applies your sedscr to an input file, creates a temporary file, then copies that file over the original. runsed has several safety checks:
It won't edit the sed script file (if you accidentally include sedscr on the command line).
It complains if you try to edit an empty file or something that isn't a file (like a directory).
If the sed script doesn't produce any output, runsed aborts instead of emptying your original file.
runsed only modifies a file if your sedscr made edits. So, the file's timestamp (Section 8.2) won't change if the file's contents weren't changed.
Like checksed, runsed expects to find a sed script named sedscr in the directory where you want to make the edits. Supply the name or names of the files to edit on the command line. Of course, shell metacharacters (Section 33.2) can be used to specify a set of files:
$ runsed *.html runsed: editing home.html: runsed: done with home.html runsed: editing new.html: runsed: done with new.html runsed: all done
runsed does not protect you from imperfect editing scripts. You should use checksed first to verify your changes before actually making them permanent with runsed. (You could also modify runsed to keep backup copies of the original versions.)
DD, JP, and TOR
A sed command can specify zero, one, or two addresses. An address can be a line number, a line addressing symbol, or a regular expression (Section 32.4) that describes a pattern.
If no address is specified, the command is applied to each line.
If there is only one address, the command is applied to any line matching the address.
If two comma-separated addresses are specified, the command is performed on the first matching line and all succeeding lines up to and including a line matching the second address. This range may match multiple times throughout the input.
If an address is followed by an exclamation mark (!), the command is applied to all lines that do not match the address.
To illustrate how addressing works, let's look at examples using the delete command, d. A script consisting of simply the d command and no address:
d
produces no output since it deletes all lines.
When a line number is supplied as an address, the command affects only that line. For instance, the following example deletes only the first line:
1d
The line number refers to an internal line count maintained by sed. This counter is not reset for multiple input files. Thus, no matter how many files were specified as input, there is only one line 1 in the input stream.
Similarly, the input stream has only one last line. It can be specified using the addressing symbol, $. The following example deletes the last line of input:
$d
The $ symbol should not be confused with the $ used in regular expressions, where it means the end of the line.
When a regular expression is supplied as an address, the command affects only the lines matching that pattern. The regular expression must be enclosed by slashes (/). The following delete command:
/^$/d
deletes only blank lines. All other lines are passed through untouched.
If you supply two addresses, you specify a range of lines over which the command is executed. The following example shows how to delete all lines surrounded by a pair of XHTML tags, in this case,
 and
, that mark the start and end of an unordered list:
/^
/,/^<\/ul>/d
It deletes all lines beginning with the line matched by the first pattern up to and including the line matched by the second pattern. Lines outside this range are not affected. If there is more than one list (another pair of
 and
 after the first), those lists will also be deleted.
The following command deletes from line 50 to the last line in the file:
50,$d
You can mix a line address and a pattern address:
1,/^$/d
This example deletes from the first line up to the first blank line, which, for instance, will delete the header from an email message.
You can think of the first address as enabling the action and the second address as disabling it. sed has no way of looking ahead to determine if the second match will be made. The action will be applied to lines once the first match is made. The command will be applied to all subsequent lines until the second match is made. In the previous example, if the file did not contain a blank line, then all lines would be deleted.
An exclamation mark following an address reverses the sense of the match. For instance, the following script deletes all lines except those inside XHTML unordered lists:
/^
/,/^<\/ul>/!d
Curly braces ({}) let you give more than one command with an address. For example, to search every line of a list, capitalize the word Caution on any of those lines, and delete any line with
:
/^
/,/^<\/ul>/{ s/Caution/CAUTION/g /
/d }
DD
Combining a series of edits in a script can have unexpected results. You might not think of the consequences one edit can have on another. New users typically think that sed applies an individual editing command to all lines of input before applying the next editing command. But the opposite is true. sed applies every editing command to the first input line before reading the second input line and applying the editing script to it. Because sed is always working with the latest version of the original line, any edit that is made changes the line for subsequent commands. sed doesn't retain the original. This means that a pattern that might have matched the original input line may no longer match the line after an edit has been made.
Let's look at an example that uses the substitute command. Suppose someone quickly wrote the following script to change pig to cow and cow to horse:
s/pig/cow/ s/cow/horse/
The first command would change pig to cow as expected. However, when the second command changed cow to horse on the same line, it also changed the cow that had been a pig. So, where the input file contained pigs and cows, the output file has only horses!
This mistake is simply a problem of the order of the commands in the script. Reversing the order of the commands changing cow into horse before changing pig into cow does the trick.
Another way to deal with this effect is to use a pattern you know won't be in the document except when you put it there, as a temporary placeholder. Either way, you know what the "document" looks like after each step in the program.
s/pig/cXXXoXXXw/ s/cow/horse/ s/cXXXoXXXw/cow/
Some sed commands change the flow through the script. For example, the N command (Section 34.16) reads another line into the pattern space without removing the current line, so you can test for patterns across multiple lines. Other commands tell sed to exit before reaching the bottom of the script or to go to a labeled command. sed also maintains a second temporary buffer called the hold space. You can copy the contents of the pattern space to the hold space and retrieve it later. The commands that make use of the hold space are discussed in Section 34.14 and other articles after it.
DD
I find that when I begin to tackle a problem using sed, I do best if I make a mental list of all the things I want to do. When I begin coding, I write a script containing a single command that does one thing. I test that it works, then I add another command, repeating this cycle until I've done all that's obvious to do. I say what's obvious because my list is not always complete, and the cycle of implement-and-test often adds other items to the list. Another approach involves actually typing the list of tasks into a file, as comments, and then slowly replacing them with sed commands. If you're one of the rare but highly appreciated breed that actually documents their code, you can just leave the comments in the script or expand on them.
It may seem to be a rather tedious process to work this way, and indeed there are a number of scripts where it's fine to take a crack at writing the whole script in one pass and then begin testing it. However, the one-step-at-a-time method is highly recommended for beginners, because you isolate each command and get to easily see what is working and what is not. When you try to do several commands at once, you might find that when problems arise, you end up recreating the recommended process in reverse; that is, removing or commenting out commands one by one until you locate the problem.
DD
Whether in sed or vi, when using the substitution command, a delimiter is required to separate the search pattern from the replacement string. The delimiter can be any character except blank or a newline (vi seems to be more restrictive than sed, although vim is extremely flexible). However, the usual practice is to use the slash (/) as a delimiter (for example, s/search/replacement/).
When either the search pattern or the replacement string contains a slash, it is easier to change the delimiter character than to escape the slash. Thus, if the pattern was attempting to match Unix pathnames, which contain slashes, you could choose another character, such as a colon, as the delimiter:
s:/usr/mail:/usr2/mail:
Note that the delimiter appears three times and is required after the replacement. Regardless of which delimiter you use, if it does appear in the search pattern or the replacement, put a backslash (\) before it to escape it.
If you don't know what characters the search pattern might have (in a shell program that handles any kind of input, for instance), the safest choice for the delimiter can be a control character.
You can use any delimiter for a pattern address (not just a slash). Put a backslash before the first delimiter. For example, to delete all lines containing /usr/mail, using a colon (:) as the delimiter:
\:/usr/mail:d
DD and JP
The backslash (\) in the replacement string of the sed substitution command is generally used to escape other metacharacters, but it is also used to include a newline in a replacement string.
Given the following input line where each item is separated by a tab:
Column1 Column2 Column3 Column4
we can replace the second tab character on each line with a newline character:
s/TAB/\ /2
Note that no spaces are permitted after the backslash. This script produces the following result:
Column1 Column2 Column3 Column4
Another example comes from the conversion of a file for troff to HTML. It converts the following line for troff:
.Ah "Major Heading"
to a similar line for HTML:

Major Heading

The twist in this problem is that the line needs to be preceded and followed by a blank line. It is an example of writing a multiline replacement string:
/^\.Ah/{ s/\.Ah */\ \
 s/"//g s/$/
\ / }
The first substitute command replaces .Ah with two newlines and
. Each backslash at the end of the line is necessary to escape the newline. The second substitution removes the quotation marks. The last command matches the end of line in the pattern space (not the embedded newlines) and adds a close h1 tag and a newline after it.
DD
As a metacharacter, the ampersand (&) represents the extent of the pattern match, not the line that was matched. For instance, you might use it to match a word and surround it with troff requests. The following example surrounds a word with point-size requests:
s/UNIX/\\s-2&\\s0/g
Because backslashes are also replacement metacharacters, two backslashes are necessary to output a single backslash. The & in the replacement string refers to the string which was originally matched, UNIX. If the input line is:
on the UNIX Operating System.
the substitute command produces:
on the \s-2UNIX\s0 Operating System.
The ampersand is particularly useful when the regular expression matches variations of a word. It allows you to specify a variable replacement string that corresponds to what was actually matched. For instance, let's say that you wanted to surround with parentheses any cross reference to a numbered section in a document. In other words, any reference such as See Section 1.4 or See Section 12.9 should appear in parentheses, as (See Section 12.9). A regular expression can match the different combination of numbers, so we use & in the replacement string and surround whatever was matched:
s/See Section [1-9][0-9]*\.[1-9][0-9]*/(&)/
The ampersand makes it possible to reference the entire match in the replacement string.
In the next example, the backslash is used to escape the ampersand, which appears literally in the replacement section:
s/ORA/O'Reilly \& Associates, Inc./g
It's easy to forget about the ampersand appearing literally in the replacement string. If we had not escaped it in this example, the output would have been O'Reilly ORA Associates, Inc.
DD
In sed, the substitution command provides metacharacters to select any individual portion of a string that is matched and recall it in the replacement string. A pair of escaped parentheses are used in sed to enclose any part of a regular expression and save it for recall. Up to nine "saves" are permitted for a single line. \n is used to recall the portion of the match that was saved, where n is a number from 1 to 9 referencing a particular "saved" string in order of use. (Section 32.13 has more information.)
For example, when converting a plain-text document into HTML, we could convert section numbers that appear in a cross-reference into an HTML hyperlink. The following expression is broken onto two lines for printing, but you should type all of it on one line:
s/\([sS]ee \)\(Section \)\([1-9][0-9]*\)\.\([1-9][0-9]*\)/ \1\2\3.\4<\/a>/
Four pairs of escaped parentheses are specified. String 1 captures the word see with an upper- or lowercase s. String 2 captures the section number (because this is a fixed string, it could have been simply retyped in the replacement string). String 3 captures the part of the section number before the decimal point, and String 4 captures the part of the section number after the decimal point. The replacement string recalls the first saved substring as \1. Next starts a link where the two parts of the section number, \3 and \4, are separated by an underscore (_) and have the string SEC- before them. Finally, the link text replays the section number again this time with a decimal point between its parts. Note that although a dot (.) is special in the search pattern and has to be quoted with a backslash there, it's not special on the replacement side and can be typed literally. Here's the script run on a short test document, using checksed (Section 34.4):
% checksed testdoc ********** < = testdoc > = sed output ********** 8c8 < See Section 1.2 for details. --- > See Section 1.2 for details. 19c19 < Be sure to see Section 23.16! --- > Be sure to see Section 23.16!
We can use a similar technique to match parts of a line and swap them. For instance, let's say there are two parts of a line separated by a colon. We can match each part, putting them within escaped parentheses and swapping them in the replacement:
% cat test1 first:second one:two % sed 's/\(.*\):\(.*\)/\2:\1/' test1 second:first two:one
The larger point is that you can recall a saved substring in any order and multiple times. If you find that you need more than nine saved matches, or would like to be able to group them into matches and submatches, take a look at Perl.
Section 43.10, Section 31.10, Section 10.9, and Section 36.23 have examples.
DD and JP
One of the more unusual options of sed's substitution command is the numeric flag that allows you to point to one particular match when there are many possible matches on a particular line. It is used where a pattern repeats itself on a line and the replacement must be made for only one of those occurrences by position. For instance, a line, perhaps containing tbl input, might contain multiple tab characters. Let's say that there are three tabs per line, and you'd like to replace the second tab with >. The following substitute command would do it:
s/TAB/>/2
TAB represents an actual tab character, which is otherwise invisible on the screen. If the input is a one-line file such as the following:
Column1TABColumn2TABColumn3TABColumn4
the output produced by running the script on this file will be:
Column1TABColumn2>Column3TABColumn4
Note that without the numeric flag, the substitute command would replace only the first tab. (Therefore, 1 can be considered the default numeric flag.) The range of the allowed numeric value is from 1 to 512, though this may be implementation-dependent.
DD
The transform command (y) is useful for exchanging lowercase letters for uppercase letters on a line. Effectively, it performs a similar function to tr (Section 21.11). It replaces any character found in the first string with the equivalent character in the second string. The command:
y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/
will convert any lowercase letter into the corresponding uppercase letter. The following:
y/abcdefghijklmnopqrstuvwxyz/nopqrstuvwxyzabcdefghijklm/
would perform a rot13 transformation a simple form of encryption in which each alphabetic character is replaced by the character halfway through the alphabet. (rot13 encryption is sometimes used to keep offensive (or comical) news postings (Section 1.21) from being read except by someone who really means to (such as if you have read the joke and now want to read the punch line). Encryption and decryption are automatically supported by most news readers, but it's fun to see how simple the encryption is. By the way, the command above handles only lowercase letters; if we'd shown uppercase as well, the command would have run past the margins!)
TOR
The pattern space is a buffer that contains the current input line. There is also a set-aside buffer called the hold space . The contents of the pattern space can be copied to the hold space, and the contents of the hold space can be copied to the pattern space. A group of commands allows you to move data between the hold space and the pattern space. The hold space is used for temporary storage, and that's it. Individual commands can't address the hold space or alter its contents.
The most frequent use of the hold space is to have it retain a duplicate of the current input line while you change the original in the pattern space. [It's also used as a way to do the "move" and "copy" commands that most editors have but which sed can't do directly because it's designed for editing a stream of input text line-by-line. GU] The commands that affect the hold space are:
|
Hold |
h |
Copy contents of pattern space to hold space, replacing previous. |
|
H |
Append newline, then append contents of pattern space, to hold space. |
|
|
Get |
g |
Copy contents of hold space to pattern space, replacing previous. |
|
G |
Append newline, then append contents of hold space, to pattern space. |
|
|
Exchange |
x |
Swap contents of hold space and pattern space. |
Each of these commands can take an address that specifies a single line or a range of lines. The hold commands (h, H) move data into the hold space and the get commands (g, G) move data from the hold space back into the pattern space. The difference between the lowercase and uppercase versions of the same command is that the lowercase command overwrites the contents of the target buffer, while the uppercase command appends to the existing contents after adding a newline.
The hold command replaces the contents of the hold space with the contents of the pattern space. The get command replaces the contents of the pattern space with the contents of the hold space. The Hold command puts a newline followed by the contents of the pattern space after the contents of the hold space. (The newline is appended to the hold space even if the hold space is empty.) The Get command puts a newline followed by the contents of the hold space after the contents of the pattern space.
The exchange command (x) swaps the contents of the two buffers. It has no side effects on either buffer.
Here's an example to illustrate putting lines into the hold space and retrieving them later. We are going to write a script that reads a particular HTML file and copies all headings to the end of the file for a summary. The headings we want start with
 or
. For example:
...
Introduction
 The blah blah blah
Background of the Project
 ...
The First Year
 ...
The Second Year
 ...
The object is to copy those headings into the hold space as sed reads them. When sed reaches the end of the body (at the  tag), output Summary:, then output the saved tags without the heading tags (
 or
).
Look at the script:
/^<h[12]>/H /^<\/body>/ { i\ Summary: x G s/<\/*h[12]>//g }
Any line matching
 or
 is added to the hold space. (All those lines are also printed; that's the default in sed unless lines have been deleted.[1]) The last part of the script watches for the  tag. When it's reached, sed inserts the Summary: heading. Then the script uses x to exchange the pattern space (which has the  tag) with the saved headers from the hold space. Now the pattern space has the saved headers. Next, G adds the  tag to the end of the headers in the pattern space. Finally, a substitute command strips the
,
,
, and
 tags. At the end of the script, the pattern space is printed by default.
The sequence of x followed by G is a way to find a matching line in this case,  and insert the contents of the hold space before the matched line. In other words, it's like an i command that inserts the hold space at the current line.
The script could do more cleanup and formatting. For instance, it could make the saved headings into a list with
 and
. But this example is mostly about the hold space.
Here's the result of running the script on the sample file:
% sed -f sedscr report.html ...
Introduction
 The blah blah blah
Background of the Project
 ...
The First Year
 ...
The Second Year
 ... Summary: Introduction Background of the Project The First Year The Second Year
For other scripts that use the hold space, see Section 34.18. For a fanciful analogy that makes clear how it works, see Section 34.17.
DD and JP
The transform command, y (Section 34.13), acts on the entire contents of the pattern space. It is something of a chore to do a letter-by-letter transformation of a portion of the line, but it is possible (though convoluted) as the following example demonstrates. [The real importance of this example is probably not the use of the y command, but the use of the hold space to isolate and preserve part of the line. TOR]
While working on a programming guide, we found that the names of statements were entered inconsistently. They needed to be uppercase, but some were lowercase while others had an initial capital letter. While the task was simple to capitalize the name of the statement there were nearly a hundred statements and it seemed a tedious project to write that many explicit substitutions of the form:
s/find the Match statement/find the MATCH statement/g
The transform command could do the lowercase-to-uppercase conversion, but it applies the conversion to the entire line. The hold space makes this task possible because we use it to store a copy of the input line while we isolate and convert the statement name in the pattern space. Look at the script first:
# capitalize statement names /the .* statement/{ h s/.*the \(.*\) statement.*/\1/ y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/ G s/\(.*\)\n\(.*the \).*\( statement.*\)/\2\1\3/ }
The address limits the procedure to lines that match the .* statement. Let's look at what each command does:
The hold command copies the current input line into the hold space. Using the sample line find the Match statement, we'll show what the contents of the pattern space and hold space contain. After the h command, the pattern space and hold space are identical.
|
Pattern space |
find the Match statement |
|
Hold space |
find the Match statement |
The substitute command extracts the name of the statement from the line and replaces the entire line with it.
|
Pattern space |
Match |
|
Hold space |
find the Match statement |
The transform command changes each lowercase letter to an uppercase letter.
|
Pattern space |
MATCH |
|
Hold space |
find the Match statement |
The Get command appends the line saved in the hold space to the pattern space. The embedded newline from the Get command is shown as \n.
|
Pattern space |
MATCH\nfind the Match statement |
|
Hold space |
find the Match statement |
The substitute command matches three different parts of the pattern space: (1) all characters up to the embedded newline, (2) all characters following the embedded newline and up to and including the followed by a space, and (3) all characters beginning with a space and followed by statement up to the end of the pattern space. The name of the statement as it appeared in the original line is matched but not saved. The replacement section of this command recalls the saved portions and reassembles them in a different order, putting the capitalized name of the command in between the and statement.
|
Pattern space |
find the MATCH statement |
|
Hold space |
find the Match statement |
Let's look at a test run. Here's our sample file:
find the Match statement Consult the Get statement. using the Read statement to retrieve data
Running the script on the sample file produces:
find the MATCH statement Consult the GET statement. using the READ statement to retrieve data
As you can see from this script, the hold space can be skillfully used to isolate and manipulate portions of the input line.
DD
Most programs that use regular expressions (Section 32.4) are able to match a pattern only on a single line of input. This makes it difficult to find or change a phrase, for instance, because it can start near the end of one line and finish near the beginning of the next line. Other patterns might be significant only when repeated on multiple lines.
sed has the ability to load more than one line into the pattern space. This allows you to match (and change) patterns that extend over multiple lines. In this article, we show how to create a multiline pattern space and manipulate its contents.
The multiline Next command, N, creates a multiline pattern space by reading a new line of input and appending it to the contents of the pattern space. The original contents of the pattern space and the new input line are separated by a newline. The embedded newline character can be matched in patterns by the escape sequence \n. In a multiline pattern space, only the metacharacter ^ matches the newline at the beginning of the pattern space, and $ matches the newline at the end. After the Next command is executed, control is then passed to subsequent commands in the script.
The Next command differs from the next command, n, which outputs the contents of the pattern space and then reads a new line of input. The next command does not create a multiline pattern space.
For our first example, let's suppose that we wanted to change "Owner and Operator Guide" to "Installation Guide", but we found that it appears in the file on two lines, splitting between Operator and Guide. For instance, here are a few lines of sample text:
Consult Section 3.1 in the Owner and Operator Guide for a description of the tape drives available on your system.
The following script looks for Operator at the end of a line, reads the next line of input, and then makes the replacement:
/Operator$/{ N s/Owner and Operator\nGuide/Installation Guide/ }
In this example, we know where the two lines split and where to specify the embedded newline. When the script is run on the sample file, it produces the two lines of output, one of which combines the first and second lines and is too long to show here. This happens because the substitute command matches the embedded newline but does not replace it. Unfortunately, you cannot use \n to insert a newline in the replacement string. You must either use the backslash to escape the newline, as follows:
s/Owner and Operator\nGuide /Installation Guide\ /
or use the \(..\) operators (Section 34.11) to keep the newline:
s/Owner and Operator\(\n\)Guide /Installation Guide\1/
This command restores the newline after Installation Guide. It is also necessary to match a blank space following Guide so the new line won't begin with a space. Now we can show the output:
Consult Section 3.1 in the Installation Guide for a description of the tape drives available on your system.
Remember, you don't have to replace the newline, but if you don't, it can make for some long lines.
What if there are other occurrences of "Owner and Operator Guide" that break over multiple lines in different places? You could change the address to match Owner, the first word in the pattern instead of the last, and then modify the regular expression to look for a space or a newline between words, as shown below:
/Owner/{ N s/Owner *\n*and *\n*Operator *\n*Guide/Installation Guide/ }
The asterisk (*) indicates that the space or newline is optional. This seems like hard work though, and indeed there is a more general way. We can read the newline into the pattern space and then use a substitute command to remove the embedded newline, wherever it is:
s/Owner and Operator Guide/Installation Guide/ /Owner/{ N s/ *\n/ / s/Owner and Operator Guide */Installation Guide\ / }
The first line of the script matches Owner and Operator Guide when it appears on a line by itself. (See the discussion at the end of the article about why this is necessary.) If we match the string Owner, we read the next line into the pattern space and replace the embedded newline with a space. Then we attempt to match the whole pattern and make the replacement followed by a newline. This script will match Owner and Operator Guide regardless of how it is broken across two lines. Here's our expanded test file:
Consult Section 3.1 in the Owner and Operator Guide for a description of the tape drives available on your system. Look in the Owner and Operator Guide shipped with your system. Two manuals are provided, including the Owner and Operator Guide and the User Guide. The Owner and Operator Guide is shipped with your system.
Running the above script on the sample file produces the following result:
% sed -f sedscr sample Consult Section 3.1 in the Installation Guide for a description of the tape drives available on your system. Look in the Installation Guide shipped with your system. Two manuals are provided, including the Installation Guide and the User Guide. The Installation Guide is shipped with your system.
In this sample script, it might seem redundant to have two substitute commands that match the pattern. The first command matches it when the pattern is found already on one line, and the second matches the pattern after two lines have been read into the pattern space. Why the first command is necessary is perhaps best demonstrated by removing that command from the script and running it on the sample file:
% sed -f sedscr2 sample Consult Section 3.1 in the Installation Guide for a description of the tape drives available on your system. Look in the Installation Guide shipped with your system. Two manuals are provided, including the Installation Guide and the User Guide.
Do you see the two problems? The most obvious problem is that the last line did not print. The last line matches Owner, and when N is executed, there is not another input line to read, so sed quits. It does not even output the line. If this is the normal behavior, the Next command should be used as follows to be safe:
$!N
It excludes the last line ($) from the Next command. As it is in our script, by matching Owner and Operator Guide on the last line, we avoid matching Owner and applying the N command. However, if the word Owner appeared on the last line, we'd have the same problem unless we implement the $!N syntax.
The second problem is a little less conspicuous. It has to do with the occurrence of Owner and Operator Guide in the second paragraph. In the input file, it is found on a line by itself:
Look in the Owner and Operator Guide shipped with your system.
In the output shown above, the blank line following shipped with your system is missing. The reason for this is that this line matches Owner and the next line, a blank line, is appended to the pattern space. The substitute command removes the embedded newline, and the blank line has in effect vanished. (If the line were not blank, the newline would still be removed but the text would appear on the same line with shipped with your system.) The best solution seems to be to avoid reading the next line when the pattern can be matched on one line. That is why the first instruction attempts to match the case where the string appears all on one line.
DD
The operations of sed's most difficult commands hold (h or H), get (g or G), and exchange (x) can be explained, somewhat fancifully, in terms of an extremely deliberate medieval scrivener or amanuensis toiling to make a copy of a manuscript. His work is bound by several spatial restrictions: the original manuscript is displayed in one room; the set of instructions for copying the manuscript are stored in a middle room; and the quill, ink, and folio are set up in yet another room. The original manuscript and the set of instructions are written in stone and cannot be moved about. The dutiful scrivener, being sounder of body than mind, is able to make a copy by going from room to room, working on only one line at a time. Entering the room where the original manuscript is, he removes from his robes a scrap of paper to take down the first line of the manuscript. Then he moves to the room containing the list of editing instructions. He reads each instruction to see if it applies to the single line he has scribbled down.
Each instruction, written in special notation, consists of two parts: a pattern and a procedure. The scrivener reads the first instruction and checks the pattern against his line. If there is no match, he doesn't have to worry about the procedure, so he goes to the next instruction. If he finds a match, the scrivener follows the action or actions specified in the procedure.
He makes the edit on his piece of paper before trying to match the pattern in the next instruction. Remember, the scrivener has to read through a series of instructions, and he reads all of them, not just the first instruction that matches the pattern. Because he makes his edits as he goes, he is always trying to match the latest version against the next pattern; he doesn't remember the original line.
When he gets to the bottom of the list of instructions, and has made any edits that were necessary on his piece of paper, he goes into the next room to copy out the line. (He doesn't need to be told to print out the line.) After that is done, he returns to the first room and takes down the next line on a new scrap of paper. When he goes to the second room, once again he reads every instruction from first to last before leaving.
This is what he normally does, that is, unless he is told otherwise. For instance, before he starts, he can be told not to write out every line (the -n option). In this case, he must wait for an instruction that tells him to print (p ). If he does not get that instruction, he throws away his piece of paper and starts over. By the way, regardless of whether or not he is told to write out the line, he always gets to the last instruction on the list.
Let's look at other kinds of instructions the scrivener has to interpret. First of all, an instruction can have zero, one, or two patterns specified:
If no pattern is specified, the same procedure is followed for each line.
If there is only one pattern, he will follow the procedure for any line matching the pattern.
If a pattern is followed by a !, the procedure is followed for all lines that do not match the pattern.
If two patterns are specified, the actions described in the procedure are performed on the first matching line and all succeeding lines until a line matches the second pattern.
The scrivener can work on only one line at a time, so you might wonder how he handles a range of lines. Each time he goes through the instructions, he tries to match only the first of two patterns. Now, after he has found a line that matches the first pattern, each time through with a new line he tries to match the second pattern. He interprets the second pattern as pattern!, so that the procedure is followed only if there is no match. When the second pattern is matched, he starts looking again for the first pattern.
Each procedure contains one or more commands or actions. Remember, if a pattern is specified with a procedure, the pattern must be matched before the procedure is executed. We have already shown many of the usual commands that are similar to other editing commands. However, there are several highly unusual commands.
For instance, the N command tells the scrivener to go, right now, and get another line, adding it to the same piece of paper. The scrivener can be instructed to "hold" on to a single piece of scrap paper. The h command tells him to make a copy of the line on another piece of paper and put it in his pocket. The x command tells him to exchange the extra piece of paper in his pocket with the one in his hand. The g command tells him to throw out the paper in his hand and replace it with the one in his pocket. The G command tells him to append the line he is holding to the paper in front of him. If he encounters a d command, he throws out the scrap of paper and begins again at the top of the list of instructions. A D command has effect when he has been instructed to append two lines on his piece of paper. The D command tells him to delete the first of those lines.
If you want the analogy converted back to computers, the first and last rooms in this medieval manor are standard input and standard output. Thus, the original file is never changed. The line on the scrivener's piece of scrap paper is in the pattern space ; the line on the piece of paper that he holds in his pocket is in the hold space. The hold space allows you to retain a duplicate of a line while you change the original in the pattern space.
Section 34.18 shows a practical application of the scrivener's work, a sed program that searches for a particular phrase that might be split across two lines.
DD
[Section 13.9 introduced a script called cgrep , a general-purpose, grep-like program built with sed. It allows you to look for one or more words that appear on one line or across several lines. This article explains the sed tricks that are necessary to do this kind of thing. It gets into territory that is essential for any advanced applications of this obscure yet wonderful editor. Section 34.14 through Section 34.17 have background information. JP]
Let's review the two examples from Section 13.9. The first command below finds all lines containing the word system in the file main.c and shows 10 additional lines of context above and below each match. The second command finds all occurrences of the word "awk" where it is followed by the word "perl" somewhere within the next 3 lines:
cgrep -10 system main.c cgrep -3 "awk.*perl"
Now the script, followed by an explanation of how it works:
case Section 35.11, expr Section 36.21, shift Section 35.22, ${?} Section 36.7, \~..~ Section 34.8, "$@" Section 35.20
#!/bin/sh # cgrep - multiline context grep using sed # Usage: cgrep [-context] pattern [file...] n=3 case $1 in -[1-9]*) n=`expr 1 - "$1"` shift esac re=${1?}; shift sed -n " 1b start : top \~$re~{ h; n; p; H; g b endif } N : start //{ =; p; } : endif $n,\$D b top " "$@"
The sed script is embedded in a bare-bones shell wrapper (Section 35.19) to parse out the initial arguments because, unlike awk and perl, sed cannot directly access command-line parameters. If the first argument looks like a -context option, variable n is reset to one more than the number of lines specified, using a little trick the argument is treated as a negative number and subtracted from 1. The pattern argument is then stored in $re, with the ${1?} syntax causing the shell to abort with an error message if no pattern was given. Any remaining arguments are passed as filenames to the sed command.
So that the $re and $n parameters can be embedded, the sed script is enclosed in double quotes (Section 27.12). We use the -n option because we don't want to print out every line by default, and because we need to use the n command in the script without its side effect of outputting a line.
The sed script itself looks rather unstructured (it was actually designed using a flowchart), but the basic algorithm is easy enough to understand. We keep a "window" of n lines in the pattern space and scroll this window through the input stream. If an occurrence of the pattern comes into the window, the entire window is printed (providing n lines of previous context), and each subsequent line is printed until the pattern scrolls out of view again (providing n lines of following context). The sed idiom N;D is used to advance the window, with the D not kicking in until the first n lines of input have been accumulated.
The core of the script is basically an if-then-else construct that decides whether we are currently "in context." (The regular expression here is delimited by tilde (~) characters because tildes are less likely to occur in the user-supplied pattern than slashes.) If we are still in context, then the next line of input is read and output, temporarily using the hold space to save the window (and effectively doing an N in the process). Else we append the next input line (N) and search for the pattern again (an empty regular expression means to reuse the last pattern). If it's now found, the pattern must have just come into view so we print the current line number followed by the contents of the window. Subsequent iterations will take the "then" branch until the pattern scrolls out of the window.
GU
The sed delete command, d, deletes the contents of the pattern space (Section 34.14) and causes a new line of input to be read, with editing resuming at the top of the script. The Delete command, D, works slightly differently: it deletes a portion of the pattern space, up to the first embedded newline. It does not cause a new line of input to be read; instead, it returns to the top of the script, applying these instructions to what remains in the pattern space. We can see the difference by writing a script that looks for a series of blank lines and outputs a single blank line. The version below uses the delete command:
# reduce multiple blank lines to one; version using d command /^$/{ N /^\n$/d }
When a blank line is encountered, the next line is appended to the pattern space. Then we try to match the embedded newline. Note that the positional metacharacters, ^ and $, match the beginning and the end of the pattern space, respectively. Here's a test file:
This line is followed by 1 blank line. This line is followed by 2 blank lines. This line is followed by 3 blank lines. This line is followed by 4 blank lines. This is the end.
Running the script on the test file produces the following result:
% sed -f sed.blank test.blank This line is followed by 1 blank line. This line is followed by 2 blank lines. This line is followed by 3 blank lines. This line is followed by 4 blank lines. This is the end.
Where there was an even number of blank lines, all the blank lines were removed. Only when there was an odd number of blank lines was a single blank line preserved. That is because the delete command clears the entire pattern space. Once the first blank line is encountered, the next line is read in, and both are deleted. If a third blank line is encountered, and the next line is not blank, the delete command is not applied, and thus a blank line is output. If we use the multiline Delete command, /^\n$/D, we get a different result, and the one that we wanted.
The reason the multiline Delete command gets the job done is that when we encounter two blank lines, the Delete command removes only the first of the two. The next time through the script, the blank line will cause another line to be read into the pattern space. If that line is not blank, both lines are output, thus ensuring that a single blank line will be output. In other words, when there are two blank lines in the pattern space, only the first is deleted. When a blank line is followed by text, the pattern space is output normally.
DD
There are two ways in sed to avoid specified portions of a document while making the edits everywhere else. You can use the ! command to specify that the edit applies only to lines that do not match the pattern. Another approach is to use the b (branch) command to skip over portions of the editing script. Let's look at an example.
We've used sed to preprocess the input to troff so that double dashes ( -- ) are converted automatically to em-dashes ( ) and straight quotes are converted to curly quotes. However, program examples in technical books are usually shown in a constant-width font that clearly shows each character as it appears on the computer screen. When typesetting a document, we don't want sed to apply the same editing rules within these examples as it does to the rest of the document. For instance, straight quotes should not be replaced by curly quotes.
Because program examples are set off by a pair of macros (something like .ES and .EE, for "Example Start" and "Example End"), we can use those as the basis for exclusion. Here's some sample text that includes an example:
.LP The \fItrue\fP command returns a zero exit status. As Joe says, "this is only useful in programming": .ES % \fBtrue\fP % \fBecho "the status was $status"\fP the status was 0 .EE
So you can say:
/^\.ES/,/^\.EE/!{ s/^"/``/ ... s/\\(em"/\\(em``/g }
All of the commands enclosed in braces ({}) will be subject to the initial pattern address.
There is another way to accomplish the same thing. The b command allows you to transfer control to another line in the script that is marked with an optional label. Using this feature, you could write the previous script like this:
/^\.ES/,/^\.EE/bend s/^"/``/ ... s/\\(em"/\\(em``/g :end
A label consists of a colon (:) followed by up to seven characters. If the label is missing, the b command branches to the end of the script. (In the example above, the label end was included just to show how to use one, but a label is not really necessary here.)
The b command is designed for flow control within the script. It allows you to create subscripts that will be applied only to lines matching certain patterns and not elsewhere. However, as in this case, it also provides a powerful way to exempt part of the text from the action of a single-level script.
The advantage of b over ! for this application is that you can more easily specify multiple conditions to avoid. The ! command can be applied to a single command or to the set of commands, enclosed in braces, that immediately follows.
TOR
The test command, t, branches to a label (or the end of the script) if a successful substitution has been made on the currently addressed line. It implies a conditional branch. Its syntax is as follows:
[address]t[label]
If no label is supplied, control falls through to the end of the script. If label is supplied, execution resumes at the line following the label.
Let's look at a spelling corrector written by Greg Ubben. The script fixes common (in this example, silly) spelling goofs; the t command tells about corrections that were made:
h s/seperate/separate/g s/compooter/computer/g s/said editor/sed editor/g s/lable/label/g t changed b : changed p g s/.*/[WAS: &]/ t
First, h (Section 34.14) holds a copy of the current input line. Then, if any of the four substitutions succeed, the command t changed branches to the corresponding label (: changed) at the end of the script. Otherwise, if no s succeeded, the b command restarts the script on the next line (as always in sed, the input line is printed before the script restarts).
After the label, the script prints the current input line (the line with a spelling error which, by now, has been corrected). Then g (Section 34.14) gets the original uncorrected line. An s command brackets that line [WAS: xxx]. Here's some sample output:
$ sed -f sedscr afile This is a separate test. [WAS: This is a seperate test.] I put a label on my computer! [WAS: I put a lable on my compooter!] That's all for now.
The final t in the script is a work-around for a bug in some versions of sed. Greg says, "The t flag is supposed to be reset after either the t command is executed or a new line of input is read, but some versions of sed don't reset it on a new line of input. So I added a do-nothing t to make sure it's reset after the previous always-true s///." Try the script without the extra t; if adding it makes the script work right, your sed has the bug and you might try a new version, like GNU sed.
JP and DD
The quit command, q, causes sed to stop reading new input lines (and stop sending them to the output). Its syntax is:
[line-address]q
It can take only a single-line address. Once the line matching address (line-address) is reached, the script will be terminated.
For instance, the following one-liner uses the quit command to print the first ten lines from a file:
% sed '10q' myfile ...
sed prints each line until it gets to line 10 and quits.
The previous version is much more efficient than its functional equivalent:
-n Section 34.3
% sed -n '1,10p' myfile
(especially if myfile is a long file) because sed doesn't need to keep reading its input once the patterns in the script are satisfied.
One possible use of q is to quit a script after you've extracted what you want from a file. There is some inefficiency in continuing to scan through a large file after sed has found what it is looking for.
TOR
The sed quit command, q (Section 34.22), is very useful for getting sed to stop processing any more input once you've done what you want.
However, you need to be very careful not to use q in any sed script that writes its edits back to the original file. After q is executed, no further output is produced. It should not be used in any case where you want to edit the front of the file and pass the remainder through unchanged. Using q in this case is a dangerous beginner's mistake.
TOR
Feeding sed (Section 34.1) newlines is easy; the real trick is getting them past the C shell and its derivatives (tcsh has the same problem, on the systems where we've tested it).
The sed documentation says that in order to insert newlines in substitute commands, you should quote them with backslashes. [Surround the commands with single quotes ('), as Chris has. If you use double quotes ("), this script will become s/foo/bar/ because of the way quoting works with backslashes and newlines (Section 27.12). JP]:
+sed -e 's/foo/b\ a\ r/'
Indeed, this works quite well in the Bourne shell and derivatives, such as bash, which do what I consider the proper thing (Section 27.12) with this input. The C shell, however, thinks it is smarter than you are and removes the trailing backslashes (Section 27.13), and instead you must type:
+sed -e 's/foo/b\\ a\\ r/'
Probably the best solution is to place your sed commands in a separate file (Section 34.3) to keep the shell's sticky fingers off them.
CT
[1] Note that this can lead to confusion when the same line is matched by several patterns and then printed, once per match!
| CONTENTS |


ISMS-P
• This Quick Start reference deployment guide provides step-by-step instructions
for deploying ISMS-P architecture-based resources and environment on the AWS cloud.
This Quick Start solution is designed to implement as many ISMS-P package controls as possible, but may not include all solutions recommended.
• Please refer to the following link for the control matrix file that describes the functions in this QuickStart.
ISMSPControlMatrix[ENG] Download
• ISMS-P stands for the integrated management system certification of information protection and personal information
protection. The check items of ISMS(Information Security Management System) and PIMS(Personal Information
Management System) that were previously operated separately are similar.
As a result of the separate operation of the two certifications, there was confusion in the company, incurring financial
and manpower burdens. In order to prevent such inconvenience, it has been integrated in accordance with related laws
(Article 47 of the “Act on Promotion of Information and Communications Network Utilization and Information
(Protection”, Article 47-3, and Article 32-3 of Personal Information Protection Act).
• Currently, there are two types of certification, ISMS and ISMS-P, and the outline is as follows.
AWS Professional Services
Security, Risk Management & Compliance Consultant (SRC)
• SCR will collaborate with customers and partners to Design, build and optimize security, DevOps , cloud migration to help
accelerate business value by providing a secure cloud operating environment.
• SRC helps protect customer environment and data with best practices, methodology for effectively manage security
risks and assist customer to develop technical competencies.
SK infosec Information Security Management System-Personal(ISMS-P)
certification consulting
• SK infosec is the No. 1 security consulting partner in Korea and has a history of exceptional performance, proven ability,
and differentiated quality that provides the best solutions and values.
• SK infosec’s security consulting has different areas of expertise digital security, compliance, information protection
management system, privacy protection, new ICT, and so on to provide the most efficient strategies
for information protection and system improvement.
• SK infosec comply with compliance by applying ISMS-P consulting service based on professional and
systematic methodology and security technology from various perspectives.