--- layout: post title: Experimenting with HTML minifier popular: popular tags: - html ---
In Optimizing HTML, I mentioned that state of HTML minifiers is rather crude at the moment. We have a large variety of JS and CSS minification tools, but almost no HTML ones. This is actually quite understandable.
First of all, minifiying scripts and stylesheets usually results in better savings, overall. Second, the nature of document markup is much more dynamic than that of scripts and styles. As a result, HTML minification has to be done “on demand”, and carries certain overhead. Only when this overhead is less then difference in time for delivering minified-vs-original document, there’s a benefit in minification. In some cases, though, savings in document size (and so bandwidth) can be more important than time spent on minification.
It’s no suprise that HTML minification is almost always a low-priority optimization. When it comes to client-side performance, there are certainly other more important things to pay attention to. Only when other aspects are taken into consideration, it is worth minifying document markup.
Few weeks ago, I decided to experiment with Javascript-based HTML minifier and created an online-based tool, with lint-like capabilities. After some tweaking, the script was able to parse and minify markup of almost any random website. The goal was to see how easy it is to implement something like this, learn HTML a bit more, and have fun in a process. Ultimately, I wanted to minify some of the popular websites and see if savings are worth all the trouble.
Today, I’d like to share this tool with you. I’ll quickly go over some of the initial features, explain how minifier works, and look into possible side effects of minification. Please note that the script is still in very early stage, and shouldn’t be used in production. If you are not interested in inner workings, feel free to skip to tests or conclusions.
At its core, minifier relies on HTML parser by John Resig. John’s parser was capable of handling quite complex documents, but would sometimes trip on some of the more obscure structures. For example, doctype declarations were not understood at all. Whenever attribute name contained characters like “-” (e.g. as in “http-equiv”), parser would fail. There were also some defficiencies in regular expressions for matching comments and CDATA sections: newlines inside them were not accounted for, so multiline comments simply weren’t matched. CDATA sections and comments inside elements with CDATA content model (e.g. SCRIPT and STYLE) were getting stripped for no apparent reason.
All of these are now fixed.
Minifier is a very small “wrapper” on top of parser. As of now it’s only about 250 LOC. It takes input string and configuration object; passes this input string to parser, and builds final output according to specified options.
For example, we can tell it to remove comments:
var input = '
baz
';
minify(input, { removeComments: true }); // '
baz
'
or to collapse boolean attributes:
var input = '<input disabled="disabled">';
minify(input, { collapseBooleanAttributes: true }); // '<input disabled>'
One of the goals I had for this little project was to have a robust test suite. HTML minifier is fully unit tested with ~100 tests at the moment. This has few benefits: anyone can change, tweak or add things without worrying to break existing functionality. It takes literally seconds to tell if script is functional in certain browser (or even in non-browser implementation, such as node.js on a server)—simply by running a test suite. Finally, tests can serve as documentation for how minifier handles some of the edge cases.
While working on minifier, I realized that oftentimes the most wasteful part of the markup is not white space, comments or boolean attributes, but inline styles, scripts, presentational or deprecated elements and attributes. None of these can be simply stripped, as that could affect state of the document and is just too obtrusive. What can be done, however, is reporting of these occurences to the user. HTMLLint is even a smaller script, whose job is exactly that—to log any deprecated or presentational elements/attributes encountered during parsing. Additionally, it detects event attributes (e.g. onclick, onmouseover, etc.). The rationale for this is that moving contents of event attributes to external script allows to take advantage of resource caching.
Before we begin, it's important to understand that minifier parses documents as HTML, not XHTML. This allows to employ such optimizations as "remove optional tags and quotes", "collapse boolean attributes", etc. Note that almost none of the options affect document validity, as per HTML 4.01. XHTML support might be added in the future, but considering that in context of pubilc web it's mostly pointless at the moment, I see little reason in doing so. Besides, minifying XHTML documents (given that they're actually served to clients properly, with "application/xhtml+xml") doesn't reduce size as much as if they were HTML.
The following is a list of current options in minifier. It is far from being exhaustive, and will most likely be extended in a future. Let’s look at each one of them quickly:
var input = '
blah
';
var output = minify(input, { removeComments: true });
output; // '
blah
'
This one should be self-explanatory. Passing truthy removeComments
tells minifier to strip HTML comments. Note that comments inside elements with CDATA content model, such as SCRIPT and STYLE, are left intact (but see next option).
var input = '';
var output = minify(input, { removeComments: true });
output; // ''
When this option is enabled, HTML comments in scripts and styles are stripped as well:
var input = '';
var output = minify(input, { removeCommentsFromCDATA: true });
output; // ''
It's worth pointing out that there's a slight difference in the way HTML comments are treated inside SCRIPT and STYLE elements. In scripts, comment start delimiter ("<!--") tells parser to ignore everything until newline:

/* treated as:
body { color: red; }
*/
Explanation of why you might want to strip comments.
This option removes CDATA sections from script and style elements:
var input = '';
var output = minify(input, { removeCDATASectionsFromCDATA: true });
output; // ''
Explanation of why you might want to do this.
This options collapses white space that contributes to text nodes in a document tree. For example:
var input = '

 foo

';
var output = minify(input, { collapseWhitespace: true });
output; // '

foo

'
It doesn’t affect significant white space; e.g. in contents of elements like SCRIPT, STYLE, PRE or TEXTAREA.
var input = '';
var output = minify(input, { collapseWhitespace: true });
output; // ''
input = '<textarea> x x x </textarea>';
output = minify(input, { collapseWhitespace: true });
output; // '<textarea> x x x </textarea>'
Now, it’s worth mentioning that this modification can have side effects, and significantly change document representation.
For example, markup like foo bar
is usually displayed as “foo bar” in browsers, with one space character in between two words. White space in markup is represented as text node in document tree. This text node’s value is a white space (e.g. U+0020
), and as long as two adjacent elements are inline-level—as they are in this example—it is this white space that contributes to a gap in between “foo” and “bar”. As soon as we remove that white space (i.e. changing markup to foobar
), representation changes from “foo bar” to “foobar”.
There are two ways to work around this issue.
First one is not to rely on such white space for document representation, and instead style elements to have margins and paddings as needed. In previous example, this could have been: foobar
(where foo class would be declared with, say, margin-right: 0.25em;
). At first, this might seem like an overkill. After all, adding class seems to defeat the purpose, resulting in larger output, when compared to a version with just one white space character. However, depending on a context, giving few elements a class for styling purposes, and then stripping white space from the entire document, can result in a smaller output.
Second option is to never fully remove white space characters, and instead always collapse them to one white space character. HTML 4.01 is actually specified to do just that, so there’s no harm in doing it upfront. Because of this, the following 2 snippets should render identically:
foo
bar
and:
foo bar
...with one space in between “foo” and “bar”. Note how in first case, there’s an entire sequence of white space characters (including line breaks).
This second option—collapsing to one white space—has not yet been added to minifier.
Another noticeable effect white space removal can have on a document is related to CSS white-space property. As I mentioned earlier, by default, adjacent sequences of white space in most of the elements collapse into one space character. But white-space
property changes it all. Some of its values result in different collapsing behavior. white-space: pre
, for example, makes whitespace render exactly as it occurs in a markup.
As a result, snippet like this:
 foo bar
renders exactly as is, and becomes:
foo bar
As of now, minifier doesn’t respect space-preserving white-space values (i.e. “pre” and “pre-wrap”). It doesn’t even understand them. Unfortunately, computing elements’ styles and determining their white-space values would be just way too complex and impractical [1]. On a bright side, it seems that white-space property is not used very often. In a future, it should be possible to add an option to minifier for specifying a way to prevent certain elements from having their content collapsed. A filtering can be based on a class, a simple selector, or maybe even by parsing element’s style attribute.
HTML 4.01 has so-called boolean attributes—“selected”, “disabled”, “checked”, etc. These may appear in a minimized (collapsed) form, where attribute value is fully ommited. For example, instead of writing <input disabled="disabled">
, we can simply write—<input disabled>
.
Minifier has an option to perform this optimization, called collapseBooleanAttributes
:
var input = '<input value="foo" readonly="readonly">';
var output = minify(input, { collapseBooleanAttributes: true });
output; // '<input value="foo" readonly>'
A potential caveat here is that if you target elements by attribute name and value, things might break after applying this optimization. Granted, this kind of case seems rather unreal, but here’s an example. If we had these rules:
input[disabled] { color: red }
input[disabled="disabled"] { color: green }
input:disabled { color: blue }
and markup like <input disabled="disabled">
, then after transforming it to <input disabled>
, second rule—input[disabled="disabled"]
—would stop matching an element. First and third ones, however, would still work as expected. I can’t imagine why someone would use this second version, and you probably won’t ever stumble upon issues like these, but it’s good to be aware of them.
By default, SGML (which HTML originates from) requires that all attribute values be delimited using either double or single quotes. But in certain cases—when attribute values contain a specific set of characters—quotes can be omitted altogether. Note that HTML specification recommends to always use quotes. There’s also an interesting explanation of why always quoting is a good idea by Jukka Korpela (although none of the dangers he’s talking about apply here). Please, use this optimization with care.
Relevant option is removeAttributeQuotes
, and it tells minifier to omit quotes when it is safe to do so:
var input = '
foo
';
var output = minify(input, { removeAttributeQuotes: true });
output; // '
foo
'
Some attributes in HTML 4.01 have default values. For example, input’s type attribute defaults to “text” and form’s method—to “get”. When enabling corresponding option in minifier (removeRedundantAttributes
), these default attribute name-value pairs get stripped from the output.
There are also few other redundancies that are taken care of as part of this optimization.
One of them is removing deprecated language attribute on SCRIPT elements. It was among markup smells I mentioned recently. Another one is coexisting “name” and “id” attributes on acnhors. And finally, redundant “javascript” labels in event handlers.
This optimization is the only one affecting document validity. That is if document is defined to be anything but HTML5 (such as HTML 4.01). When useShortDoctype
option is enabled, existing doctype is replaced with its short (HTML5) version—<!DOCTYPE html>
. As mentioned before, this replacement is generally pretty safe, but you should decide for yourself if this is something worth doing.
The corresponding option is removeEmptyAttributes
, and when enabled, all attributes with empty values are simply removed from the output. This includes blank values as well—those consisting of white space only.
var input = '
foo
';
var ouptut = minify(input, { removeEmptyAttributes: true });
output; //
foo

Note that not all “empty” attributes are removed. For example, both “src” and “alt” attributes are required on IMG elements, so we can’t remove them, even if they’re empty. Right now, only core attributes (id, class, style, title), i18n ones (lang, dir) and event ones (onclick, ondblclick, etc.) are considered “safe” for removal.
The caveat here is that, similar to “collapse boolean attributes” option, this change can affect certain style or script behavior. For example, you might want to target all elements with class attribute—*[class] { ... }
. This will apply to elements with empty class, such as
bar

, but obviously not to those without—
bar

.
This might not be a big issue, but take it into consideration.
Some elements in HTML 4.01 are allowed to have their tags omitted. Optional tags are either end one (e.g.
) or both—start and end ones (e.g.

and

). Note that start tag can never be optional on its own.
Corresponding option in minifier is removeOptionalTags
. Currently, it only strips end tags of HTML, HEAD, BODY, THEAD, TBODY and TFOOT elements. I don’t fully understand the process of creating document tree from “unclosed” markup, so I’m not sure when it’s safe to omit tags like

.
For example, I can see how removing BODY start tag can have side effects. Let’s say we have a markup like this (with omitted HTML 4.01 doctype, for brevity):

<title>x</title>

x

and the same markup with HEAD and BODY tags removed:
<title>x</title>

x

Note that second version is a perfectly valid document. It just has start and end tags of HEAD and BODY elements omitted. Now what seems to happen here, in a second version, is this:
Browser starts parsing, encounters TITLE tag, and given lack of starting HTML and HEAD tags, creates both elements implicitly (first, HTML, then HEAD as its immediate child). It then continues parsing, up until it stumbles upon P element, which, as per DTD, can not be a child of HEAD. Browser is therefore forced to implicitly close HEAD element, start BODY element, and continue parsing further.
Now, if we were to display both of these documents, first one would alert “SCRIPT” and second one—“P”. This is becase in original version, SCRIPT element is defined explicitly to be a child of BODY, and in modified version—child of HEAD (due to the way parsing works). The behavior of two documents is therefore not identical. We've got a "problem".
Just like with previous “gotchas”, I’m not sure how likely this type of scenario is to appear in real life. From what I can see, the only other element (besides SCRIPT), allowed as child of both—HEAD and BODY, is OBJECT. As for the future, it should be possible to make minifier strip other optional tags as well. But only in safe scenarios.
It’s also worth mentioning that unclosed elements can result in slightly slower parsing times. Unfortunately, there are no extensive benchmarks done on this topic, and results seem to vary across browsers.
This optimization is probably one of the most obtrusive ones, which is why it is disabled by default. Think of it as an experimental addition, and employ with great care. There are dozens of valid use cases for occurence of empty elements in a document. They can be used as placeholders for content inserted later with scripting; or for presentational purposes, such as to implement rounded corners, shadows, float clearing, etc. There are probably other cases, which I can’t think of at the moment.
When enabled, minifier simply removes all elements with empty contents (but not those with empty content model, such as IMG, LINK, or BR).
For example:
var input = '

';
var output = minify(input, { removeEmptyElements: true });
output; // ''
input = '
blah
';
var output = minify(input, { removeEmptyElements: true });
output; // '
blah
';
There are few things to be aware of. First of all, elements containing only other empty elements are not removed. For example:
var input = '

';
var output = minify(input, { removeEmptyElements: true });
output; // '

'
Note how only inner DIV element—the one with actual empty contents—is removed.
Second of all, only truly empty string is considered an empty content. This does not include spaces, newlines, or other white space characters:
var input = '

'; // note one space character in between
var output = minify(input, { removeEmptyElements: true });
output; // '

'
Also note that comments are parsed as separate entities and so don’t affect “emptiness” of elements:
var input = '

';
var output = minify(input, { removeEmptyElements: true });
output; // ''
As with other optimizations, some of these limitations will likely be removed in the future.
This option simply toggles linting. You can create new HTMLLint object and pass it to minifier. During minification, lint object silently logs all "suspicious" activity. It exposes populate
method, which accepts element and inserts its log into this element:
var lint = new HTMLLint();
minify(' some input... ', { lint: lint });
lint.populate(document.getElementById('someElement'));
So how does minifier stand against real-life markup? Let’s take a look at minification results of some of the popular websites (note that when gzip'ing documents, 6th level of compression (default) was used):
Original size: 217KB (35.8KB gzipped)
Minified size: 206.6KB (34.3KB gzipped)
Savings: 10.4KB (1.5KB gzipped)
Minifying home page of amazon.com saves about 10KB with uncompressed document, and only 1.5KB with compressed one. What’s interesting is that humongous 217KB is actually a result of miriad of inline styles and scripts scattered throughout a document. Replacing those with external scripts would be the best optimization. Getting rid of occasional style attributes would help too.
Original size: 82KB (18.2KB gzipped)
Minified size: 74.9KB (17.2KB gzipped)
Savings: 7KB (1KB gzipped)
On digg.com, reduction is slightly smaller—7KB (1KB gzipped). The markup is not as cluttered as on amazon, but still has smells: inline scripts (and unnecessary comments in them), deprecated attributes, anchors defunct without scripting, etc. The benefits of minification are rather small here.
Original size: 177.6KB (32.4KB gzipped)
Minified size: 157.3KB (29.7KB gzipped)
Savings: 20.3KB (2.7KB gzipped)
Trying out home page of ajaxian.com, we see a difference of ~20KB—even better reduction in size. And again, compressed documents show savings of only 2.7KB. Speaking of compression, ajaxian.com shamelessly serves its 177KB-large document uncompressed. There’s also some redundant markup, like unnecessary ’s, excessive style attributes, lots of comments, and few inline scripts. Removing all of those, and turning on compression would be an ultimate optimization.
Original size: 128.8KB (19.8KB gzipped)
Minified size: 89.4KB (17.1KB gzipped)
Savings: 39.4KB (2.7KB gzipped)
linkedin.com surprises with savings of almost 40KB (!) after minification. Looking at the source, we see that large number is explained by excessive amount of whitespace. This is a good example of how carelessly used whitespace can add up to huge number like this. And again, gzip saves the day; minifying compressed document reduces it only by 2.7KB.
Original size: 703KB (122.5KB gzipped)
Minified size: 572KB (106.4KB gzipped)
Savings: 131KB (16KB gzipped)
Large static documents is where HTML minification truly shines, and HTML version of ECMAScript (3rd ed.) language specification is a clear demonstration of it. Minifying document results in savings of 131KB (!) for an uncompressed document, and 16KB for compressed one. Since document is served statically, there’s hardly any reason not to apply minification here.
It’s pretty obvious that the best candidates for html minification are large static documents. Or just static documents—FAQ’s, standalone articles, etc. Anything that can’t be compressed (e.g. if there are not enough access rights, to enable gzip on a server) would benefit from minification as well. Even when serving gzipped content, it’s worth remembering that not everyone is getting gzip. So clients that are being sent gzipped content could receive 2-3KB smaller file, whereas those receiving uncompressed content could end up with files up to whopping 10-20KB smaller than original ones.
One of the biggest problems I see, when it comes to dynamic minification, is the possibility of error. The core of the issue is that minification relies on parsing, and parsing HTML is a pretty tricky business. Even though minifier applies a strict set of rules—removing quotes and optional tags only when it is absolutely safe to do so, a single misplaced character in start tag can trip parser and wreak havoc on an entire document. This is especially relevant when there’s an inclusion of user-generated content.
As an example, browsers usually understand empty end tags (allowed in HTML)—
 test </>
, but parser, which minifier is based on, would immediately choke here and stop. Another example is attributes containing “weird” characters— test
(note trailing quote after an attribute). Many browsers happily parse this element, ignoring trailing quote. But parser, once again, falls short and bails out.
It’s certainly possible to tame errors and simply output original, uncompressed document. But this brings us to another downside—time spent on minification. Even when errors are not an issue, there’s an actual overhead of parsing and processing document tree. Minifying home page of amazon.com in pretty speedy nightly webkit, for example, takes exactly 1 second. Most of that time is consumed by parsing. 1 second is quite a lot. An acceptable time for real-time minification would be somewhere around 50-100ms. This problem can be mitigated by optimizing parser, or porting script to be executed in a faster environment (v8 on a server?).
Curiously, Opera 10.50 beta (on Mac OS X) managed to beat WebKit and completed this task almost twice faster (~500ms). Unfortunately, this version suffers from some bugs in regex matching, and fails half of the test suite. Hopefully, those issues will be resolved in later revisions.
Another interesting performance observation was with V8 engine. When testing with version 1.3.x, the time it took to minify amazon.com home page was 0.6 secs. However, version <IP_ADDRESS> (currently latest stabe) performed same task in excruciatingly long 2 seconds.
I can think of many other things to improve in minifier. Unfortunately, I don’t have much time to work on it. The project is licensed under MIT, and is free for use/modification by anyone interested. Test suite should make collaboration easy. There’s a short todo list on a bottom of project page. Among other things, it lists some of the known bugs.
As always, any questions, corrections, and suggestions are very much welcomed.
Enjoy.
1. “white-space: pre” declaration could be part of a rule from within an extrnal stylesheet; getting computed style would require downloading, parsing and analyzing every single stylesheet linked from the document (or imported from within another stylesheet).


The following is a true story,
only the names have
been changed to protect the guilty
Well I left my job in my home town
And I headed for the smoke
Got a rock ‘n’ roll band
and a fast right hand
Gonna get to the top
Nothings gonna stop us – no nothing
So if you got the money
We got the sound
You put it up
and we’ll put it down
If you got the dollar
We got the song
Just wanna boogie woogie
all night long,
Yeh boogie
I’ve got holes in my shoes
I’ve got holes in my teeth
I’ve got holes in my socks
I can’t get no sleep
I’m trying to make a million
And I’ve got patches on the patches
On my old blue jeans
Well they used to be blue
When they used to be new
When they used to be clean
And I’ve got a Mama whose a hummer
Whose keeping me alive
While I’m in the band doing
drinking with the boys
She’s working 9 to 5
(knows her place that woman)
Just you wait
One of these days see me
driving round town
In my rock ‘n’ Rolls Royce
with the sun roof down
And my bottle of booze
(no summer time blues)
Shouting out ‘look at me’
In my rock ‘n’ roll voice
No it ain’t no fun
waiting round to be a millionaire


Get started
Handle errors
Handle exports
Handle Jiix import
Customize the style
Redefine the controls
Settings the attributes programmatically
Advanced example - answering a math question
MATH with REST
Custom resources
Disable the solver
Tuning the solver
Activate debug mode
Accessing ink stats


| Displaying 10 libraries (1-2 of 2) | |||||
| Library | Features | Distance | Library Type | ||
|---|---|---|---|---|---|
| 1. |
1000 Grand Avenue
|
Map It | Public |
Add as favorite | |
| 2. |
West Des Moines Public Library
4000 MILLS CIVIC PKWY
|
Map It | Public |
Add as favorite |
Only libraries that have created a profile in the WorldCat Registry are listed in the library search results.
If you are a librarian and do not see your library listed, you may create a free WorldCat Registry profile.


{% extends 'email_template.html' %} {% block content %}
Success!
Your password for Author's Haven account has been reset


{% extends "base.html" %} {% block page_title %}CWR matching result{% endblock %} {% block content %}
Here the matching results would be shown in detail.
Matches


Limericks Week in ManageIQ
WATCH ME
read me
This post in form is quite new
Again we’ve code changes to view
If you would like to listen
There’s a video submission
This post courtesy of you know who.
To my accomplice offer much gratitude
The video’s a great interlude
To give credit where due,
Thank Allen B W,
Whose assistance was greatly valued.
New code changes are easily viewed
Two week’s worth is some amplitude
Peruse them all here
We’ve been busy, it’s clear
With plentiful changes accrued.
The dialog editor’s long overlooked
But with pretty changes, we’re all really hooked.
See drag and drop,
The Ops UI pop,
Related bug fixes are booked.
Large removals are always well-greeted
Non-shared Ansible tests were deleted
If you’d like to admire,
This one will inspire
Cassette cuts to soon be completed.
Validation of provider credentials
Includes raw_connect in its essentials
The move to the queue
Needs decrypt to go through
The specifics have lots of potentials.
Default value selection was broken
In the service UI, it was spoken
A coder from Houston
Had fixes induced in
A PR that is hereby bespoken.
Some catalogue BZs have been closed
Doc notes have been newly composed
Azure items are easy
And Amazon breezy
The details have here been exposed.
Logo rendering has now been adjusted
As spacing had been somewhat busted
Docker whale resized,
The layout revised
Mobile styling no longer combusted.
Our integration test repo’s abuzz
Includes Widgetastic now it does
Some python’s included
Enhancements subtruded
Less web-widget abstracted twas.
Provider generators can be synced
From the core repo to which they’re linked
Shared specs are looked for,
Refactors we adore
We’re just thrilled when updates are succinct.
Generic Objects page has been refreshed
Feature names are hence newly addressed
Fresh sections galore
Old views are done for
You’ll find show_list is now at its best!
A coder who is from Hong Kong
Insisted credentials belong
Azure Classic renewed
So as not to exclude
With Resource Manager it’ll throng.
Metric collection has been quite updated
From sources that are not related
Not Hawkular-centered
More places we’ve ventured
Single vendors you’ll find are outdated.
Seeding timeouts are being repaired
WAN seeding no longer impaired
If you wish, take a gander
It shall not meander
Descriptive errors now are prepared.
This amusement must come to an end
Dear reader, your help we commend
If you’d like to submit
Please add a commit
Until then, the next post we attend.


In this game, you click balloons and win REAL Amazon gift cards. It's easy! Just start the game and win money...
1) I hope you figured out this game is trolling you! There are no gift cards or balloons! Games like this are fun practical coding jokes you can play on your friends. Let's make our own! Start out by picking a backdrop and making a start button sprite.
2) There are four separate things the start button should do. Trying to do all of them together is complicated, so let's work on them one at a time! The first thing the button should do is move back to the center of the screen no matter where it is. Drag your button to different places on the screen (or use a block to put it in random places), and make sure it goes back to the center.
3) The second thing the start sprite should do is only move back to the center of the screen if it's far away from the cursor. Hint: you will need a sensing block called "Distance to ____."
4) The third thing the start sprite should do is run away from the mouse cursor as long as the cursor is close. Make sure it moves real fast! You can do it! Hint: If the mouse cursor is to the right of the button, have it move left. If the mouse cursor is above the button, have it move down. Where should it move if the mouse cursor is to the left of the sprite?
5) The final thing the button should do is teleport to the opposite side if it gets pushed off screen. You got this. Hint: You can use the 'minus' operator block with a '0' to flip any positive or negative number to its opposite.
6) Great! Now we are going to work on making that button more tempting to players. Try out some of the effects you can find in the "Looks" category. Try to make the button look like it WANTS to be pressed.
7) Awesome! Now we want to make it more believable that other players (not the friend you are making play this) can play and win your game. We are going to add notifications from other players getting high scores or winning prizes. They don't actually exist of course, but your friend doesn't know that! Create a sprite, and put a message in it that a player has won a prize. Then make more costumes, as many as you can, each with a unique player name and prize.
8) Do you remember Ghostbusters? Basically, do all of the same steps as your ghost, except with your messages. They should disappear and reappear every few seconds. Each time, have them appear in a new random location and change to a different costume.
9) When each message appears, make it appear in a new color. Or, if you want to be truly annoying, add a sound effect when they appear.
10) As a final trick, make a score variable. Name it whatever you like, we won't use it. It's just to make the player think they'll get money or points. Muahahahaha, the illusion is complete!
Bonus) Did you notice how the start button is very jittery and shakes sometimes while it moves? This is because you likely coded it to only move in 8 or fewer different directions. Recode it using trigonometry to truly be able to travel in any direction. Compare how smooth this example is compared to the base version.


I earned my Ph.D. and M.S. at UC San Diego, advised by Julian McAuley. Before that, I received my B.S.E. from Princeton University in Operations Research and Financial Engineering (ORFE).
My research is centered around Natural Language Generation, Interactive AI, and Conversational Agents.
My research focuses on the intersection of natural language processing (NLP) and recommender systems. I explore personalization in natural language generation and question-answering, subjectivity in knowledge bases, and mining the wealth of data available publicly online. My previous research includes structured text generation, sentiment analysis, and spatial clustering in neuroscience.
My research is centered around Natural Language Generation, Interactive AI, and Conversational Agents.
I also focus on making large-scale, useful and interesting datasets available for research, hosted at my pages on Kaggle and HuggingFace.
I have been a PC/reviewer for: ICLR, NeurIPS, ICML, EMNLP, NAACL-HLT, WWW, KDD, RecSys, AAAI, and ACL Rolling Review.
Selected reseach projects below. For a complete list, see my Google Scholar page.
Quick links to sections: publications, workshop/other projects.
|
Assistive Recipe Editing through Critiquing
We present RecipeCrit, a hierarchical denoising auto-encoder that edits recipes given ingredient-level critiques. The model is trained for recipe completion to learn semantic relationships within recipes. Our work's main innovation is our unsupervised critiquing module that allows users to edit recipes by interacting with the predicted ingredients; the system iteratively rewrites recipes to satisfy users' feedback. Experiments on the Recipe1M recipe dataset show that our model can more effectively edit recipes compared to strong language-modeling baselines, creating recipes that satisfy user constraints and are more correct, serendipitous, coherent, and relevant as measured by human judges. |
|
|
SHARE: a System for Hierarchical Assistive Recipe Editing
We introduce SHARE: a System for Hierarchical Assistive Recipe Editing to assist home cooks with dietary restrictions---a population under-served by existing cooking resources. Our hierarchical recipe editor makes necessary substitutions to a recipe's ingredients list and re-writes the directions to make use of the new ingredients. We show that recipe editing is a challenging task that cannot be adequately solved with human-written ingredient substitution rules or straightforward adaptation of state-of-the-art models for recipe generation. |
|
|
Self-Supervised Bot Play for Conversational Recommendation with
Rationales
We introduce a framework for training interactive recommender systems for multi-turn conversational recommendation using self-supervised bot-play predicated on review data. We demonstrate via experiments on three real-world datasets that our method is model-agnostic and allows simple matrix factorization and linear recommender systems to out-perform state-of-the-art existing techniques for conversational recommendation. |
|
|
Instilling type knowledge in language models via multi-task QA
We introduce a novel multi-task QA framework for pre-training language models on joint articles and knowledge bases to recognize types, entities, and their relationship in a type ontology. We demonstrate that generative language models trained using our framework can a) achieve up to 14.9% (absolute) / 49.4% (relative) improvement in zero-shot dialog state tracking (DST) over current state-of-the-art, b) infer types for previously unseen entities better than other pre-trained language models (+16.7 F1), and c) discover novel complex and meaningful types. We also release the WikiWiki dataset linking 10M Wikipedia articles with 41K unique types from the Wikidata knowledge graph. |
|
|
Zero-shot Generalization in Dialog State Tracking through Generative Question
Answering
We introduce a novel ontology-free framework that supports natural language queries for unseen constraints and slots in multi-domain task-oriented dialogs. Our approach is based on generative question-answering using a conditional language model pre-trained on substantive English sentences. Our model improves joint goal accuracy in zero-shot domain adaptation settings by up to 9\% (absolute) over the previous state-of-the-art on the MultiWOZ 2.1 dataset. |
|
|
Interview: Large-scale Modeling of Media Dialog with Discourse Patterns and
Knowledge Grounding
We perform the first large-scale analysis of discourse in media dialog and its impact on generative modeling of dialog turns, with a focus on interrogative patterns and use of external knowledge. We introduce Interview—a large-scale (105K conversations) media dialog dataset collected from news interview transcripts—which allows us to investigate such patterns at scale. |
|
|
Speech Recognition and Multi-Speaker Diarization of Long Conversations
We compare separate and joint frameworks for speech recognition (ASR) and speaker diarization in a multi-speaker setting, showing that joint models can perform well in settings without known utterance bounds. We release a dataset to support multi-speaker ASR and diarization, drawn from a weekly radio program: This American Life. |
|
|
Interview: A Large-Scale Open-Source Corpus of Media Dialog
We introduce a large-scale dataset of media dialog: interview-style transcripts from NPR talk shows. We demonstrate its usefulness for dialog modeling and its special traits and dialog structure when compared to existing small-scale spontaneous dialog transcriptions and written proxies for conversation. |
|
|
Generating Personalized Recipes from Historical User Preferences
We propose the task of personalized recipe generation: expanding a name and incomplete ingredient details into a complete natural-text recipe instruction set, aligned with a user's historical preferences; we present a model that attends on user activity traces to solve this task. |
|
|
Expression of the transient receptor potential channels TRPV1, TRPA1 and TRPM8 in
mouse trigeminal primary afferent neurons innervating the dura
We study the size and clustering of TRP-channel expressing neurons in the trigeminal dura. Our results suggest that TRPV1 and TRPA1 but not TRPM8 channels likely contribute to the excitation of dural afferent neurons and the subsequent activation of the headache circuit. These results provide an anatomical basis for understanding further the functional significance of TRP channels in headache pathophysiology. |
|
Variable Bitrate Discrete Neural Representations via Causal Self-Attention
Generative modeling across text, video, and image domains has benefited from the introduction of discrete (quantized) representations for continuous data. We aim to learn a single model able to produce discrete representations at different granularities (bitrates). We propose a framework based on the Perceiver IO architecture, incorporating causal attention to learn ordered latent codes that can then be adaptively pruned to a target compression rate. |
|
|
Bernard: A Stateful Neural Open-domain Socialbot
We propose Bernard: a framework for an engaging open-domain socialbot. We explore various strategies to generate coherent dialog given an arbitrary dialog history. We incorporate a stateful autonomous dialog manager using non-deterministic finite automata to control multi-turn conversations. |
|
|
Recipes for Success: Data Science in the Home Kitchen
We survey the history of data science and machine learning methods to assist home cooks find and make nutritious, delicious meals. This article explores domains including recipe recommendation, recipe generation, image-to-recipe retrieval, and assistive technologies that integrate with the cooking process. |
|
|
Exploring Rich Features for Sentiment Analysis with Various Machine Learning
Models
This was my senior thesis at Princeton University, supervised by Dr. Xiaoyan Li. Here, we investigate the use of rich features to extend the bag-of-words model for sentiment analysis using machine learning in the movie review domain. We focus on subjectivity analysis and sentence position features. In addition, I created a manually labeled set of subjective and summary sentences for 2000 reviews in the Cornell IMDB movie review corpus. |
|
|
Learning Rate Analysis for Temporal-Difference Learning
The objective was to find optimal policies in an energy allocation problem. The model contained a battery, power grid, wind energy, and demand, with the latter three stochastic variables. I investigated the performance of Q-Learning and SARSA given different learning rates. |
I have worked as a research scientist and software engineer in the tech and finance industries. My work focuses on applying AI and machine learning to improve consumer products (via personalization, knowledge grounding, and data quality) and democratizing access.
My work is centered around Natural Language Generation, Interactive AI, and Conversational Agents.
|
Meta: Natural Language Processing
Heterogeneous Knowledge Graphs, Text Generation (August 2022 - Present) |
|
|
Salesforce
AI: Natural Language Processing
Meta-Learning for Low-Resource Dialog Agents (October 2021 - February 2022) |
|
|
Amazon Alexa: Alexa
Natural Language Understanding
Concept-centric Pre-training for Language Modeling (June 2021 - October
2021)
Generalizable Dialog State Tracking (June 2020 - October 2020) |
|
|
Google Cloud AI: Kaggle
Dataset Categorization and Search (June 2019 - September 2019) |
|
|
Bloomberg: Structured Products
Data Quality Control Platform (June 2017 - September 2018)
Semi-Structured Text Clustering for Securities (June 2017 - September
2018) |
|
|
Goldman
Sachs: Operations Analytics Strats
Machine Learning Platform as a Service (July 2016 - June 2017)
FIX Market Data Pipeline (July 2016 - August 2017) |
|
|
Goldman
Sachs: Operations Analytics Strats
Automated Invoice Recognition and Extraction (June 2015 - August 2015)
Workflow Assignment through Mixed Integer Linear Programming (June 2015 - August
2015) |
A simple game inspired by Haruki Murakami's Hard-Boiled Wonderland and
the End of the World. Move around the world with the arrow keys.
Using cellular automata to generate mazes. Allows customization of maze
dimensions and visualizations. Solve the maze with arrow keys. Contains an auto-solver
feature.
Set a beginning configuration and a ruleset, and allow a cellular automaton
to grow. Allows flipping of grid squares as the automaton is running, as well as
customization of run speed and canvas dimensions.
Break cookies into different-sized pieces to feed a friendly colony of ants,
as they're constantly harrassed by locusts. Experiment with HTML5 canvas.
Dabbling in development of an incremental RPG. Progress through an infinite
dungeon and collect party members and upgrade your gear.
In my work with the Structured Products team (Mortgage Waterfall
Infrastructure), I helped to design and implement a Spark-based infrastructure for high
bandwidth data processing jobs. We wrote a series of applications to wrap common
data-access, filtering, and testing paradigms, so other members of the Structured Products
group could write test logic as a self-contained plugin and farm the job out to our Spark
cluster. We achieved a 100x+ speedup for some jobs due to parallelization, and abstracted
away Spark application boilerplate.
The platform infrastructure was written in Python with jobs distributed via pySpark.
As part of my work with the Quality Control team in Mortgage Waterfall
Infrastructure, I am currently investigating how various mortgage-backed securities (pools,
generics, CMOs) can be clustered by the shape of the data we regularly receive for each
security. This entails identifying data shape features for semi-structured text data
received for over 2.5 million securities and clustering on time-series data for each
security.
The application was written in Python, using Spark
MLlib library for feature processing and clustering, and PostgreSQL to store time series data, accessed
programmatically via SQLAlchemy. The application
utilizes the infrastructure framework developed as part of the Data QC Platform.
This was my primary project in Ops Analytics Strats (OAS), in the Operations
Technology group at Goldman Sachs. Over the course of 10 months, we built a platform for
data science and machine learning on top of the firm's centralized data store. Our goal was
to allow anybody to classify and regress on arbitrary datasets without needing a deep
background in programming for machine learning. One client was an internal data platforms
team, which used our platform to better predict expected runtimes for their data ingestion
workflow.
The platform infrastructure was written in Java. For machine learning, we used Spark MLlib and R.
This was the project team I worked on for Grey Wolf on Goldman Sachs Asset
Management - Fixed Income (GSAM FI) as a new Technology Analyst in the fall of 2016. Over 6
weeks, our team built a system to consume FIX messages from several marketplaces, parse them
into a standardized object, and store the messages in a database. Our application allowed
the GSAM FI team to consume a vastly greater quantity of market messages and made it
available for further analysis by traders. It is currently a production system.
The application was written in Java, using the QuickFIX/J library for FIX message decoding.
This was my main project as a Summer Technology Analyst in Operations
Analytics Strats. We worked with Accounting Services to build a tool that could
automatically index certain values from images of invoices. They received invoices through
email or fax and scanned them into .tiff files. Our algorithm relied on segmentation of
individual values to produce structured output and templating to identify likely segment
locations.
The program was written in Python, using the numpy
library for fast array operations and scikit-image
library for image processing. We used the Tesseract open-source OCR engine.
This was another project that I worked on as a Summer Technology Analyst in
Operations Analytics Strats. We worked with several different Operations groups to produce a
tool for automatically assigning tasks to available analysts. This took into account
available analysts, their expected bandwidth for the remainder of the day, their proficiency
with incoming tasks, and implemented a Maker-Checker process. The tool is to be run by
managers, and outputs a list of each person's assignments as well as a list of unassigned
tasks with reasoning listed.
The program was written in R, using the Rglpk library for
linear programming.
This was my senior thesis at Princeton University, supervised by Dr. Xiaoyan
Li. Here, we investigate the use of rich features to extend the bag-of-words model for
sentiment analysis using machine learning in the movie review domain. We focus on
subjectivity analysis and sentence position features. In addition, I created a manually
labeled set of subjective and summary sentences for 2000 reviews in the Cornell IMDB movie
review corpus. The full title of the thesis is Exploring Rich Features for Sentiment
Analysis with Various Machine Learning Models.
All model training and execution were performed in Python, using NLTK and TextBlob for document parsing and
feature extraction, and scikit-learn for
machine learning algorithms. Data analysis was done in R.
Documents: Poster
presented at the 2016 IEEE MIT Undergraduate Research Technology Conference. A full copy of
the thesis may be requested via Princeton DataSpace. You
can also download a .7z archive of my manually labeled movie reviews.
I worked on this semester-long project in Junior year of college for EGR 392
- Creativity, Innovation, and Design. We helped the campus career center provide resources
for career exploration and preparation in a way that was better suited for the undergraduate
population. We created a regular mentorship program for underclassmen and upperclassmen who
had experience with various research and industry roles. We also held a well-attended career
exploration workshop on campus.
The team consisted of Grace Chang '17, Catherine Idylle '16, Jean Choi '15, Maggie Zhang
'16, Annie Chen '15, and myself.
Documents: Presentation given to
the EGR 392 class, as well as administrators of Princeton career services.
This was a project that I worked on for ORF 418: Optimal Learning in Spring
2014. I worked with Max Kaplan. We created an algorithm for pricing Express Lanes on the
I-110 Freeway using Optimal Learning techniques. Specifically, we tested the Knowledge
Gradient, Interval Estimation, Pure Exploitation, and Constrained Exploration algorithms
with linear and logistic belief models.
Code and graphics done in MATLAB.
Documents: Project Report, Project Presentation


--- redirect_from: /docs/1.0.0-alpha.3/classes/url.urlmatcher.html ---
A UrlMatcher
instance to append as a child of the current UrlMatcher
.
The URL path to match, e.g. $location.path()
.
URL search parameters, e.g. $location.search()
.
URL hash e.g. $location.hash()
.
The captured parameter values.
the values to substitute for the parameters in this pattern.
the formatted URL (path and optionally search part).
The object hash of parameters to validate.
Returns true
if params
validates, otherwise false
.
Given a matcher, return an array with the matcher's path segments and path params, in order
Given a matcher, return an array with the matcher's path segments and path params, in order
Given a matcher, return an array with the matcher's query params
Given a matcher, return an array with the matcher's query params
Generated using TypeDoc
object
ui.router.util.type:UrlMatcher
Matches URLs against patterns and extracts named parameters from the path or the search part of the URL. A URL pattern consists of a path pattern, optionally followed by '?' and a list of search parameters. Multiple search parameter names are separated by '&'. Search parameters do not influence whether or not a URL is matched, but their values are passed through into the matched parameters returned by {@link ui.router.util.type:UrlMatcher#methods_exec exec}.
Path parameter placeholders can be specified using simple colon/catch-all syntax or curly brace syntax, which optionally allows a regular expression for the parameter to be specified:
':'
name - colon placeholder'*'
name - catch-all placeholder'{' name '}'
- curly placeholder'{' name ':' regexp|type '}'
- curly placeholder with regexp or type name. Should the regexp itself contain curly braces, they must be in matched pairs or escaped with a backslash.Parameter names may contain only word characters (latin letters, digits, and underscore) and must be unique within the pattern (across both path and search parameters). For colon placeholders or curly placeholders without an explicit regexp, a path parameter matches any number of characters other than '/'. For catch-all placeholders the path parameter matches any number of characters.
Examples:
'/hello/'
- Matches only if the path is exactly '/hello/'. There is no special treatment for trailing slashes, and patterns have to match the entire path, not just a prefix.'/user/:id'
- Matches '/user/bob' or '/user/1234!!!' or even '/user/' but not '/user' or '/user/bob/details'. The second path segment will be captured as the parameter 'id'.'/user/{id}'
- Same as the previous example, but using curly brace syntax.'/user/{id:[^/]*}'
- Same as the previous example.'/user/{id:[0-9a-fA-F]{1,8}}'
- Similar to the previous example, but only matches if the id parameter consists of 1 to 8 hex digits.'/files/{path:.*}'
- Matches any URL starting with '/files/' and captures the rest of the path into the parameter 'path'.'/files/*path'
- ditto.'/calendar/{start:date}'
- Matches "/calendar/2014-11-12" (because the pattern defined in the built-indate
Type matches2014-11-12
) and provides a Date object in $stateParams.startThe pattern to compile into a matcher.
A configuration object hash
caseInsensitive
-true
if URL matching should be case insensitive, otherwisefalse
, the default value (for backward compatibility) isfalse
.strict
-false
if matching against a URL with a trailing slash should be treated as equivalent to a URL without a trailing slash, the default value istrue
.{string} prefix A static prefix of this pattern. The matcher guarantees that any URL matching this matcher (i.e. any string for which {@link ui.router.util.type:UrlMatcher#methods_exec exec()} returns non-null) will start with this prefix.
{string} pattern The pattern that was passed into the constructor
New
UrlMatcher
object


Thank you so much for thinking of us!
Your love, support and presence at our wedding is all we need.
But if you would like to give us a gift, please vote on our honeymoon.


{% extends "header.html" %} {% block body %}
In this video we'll be discussing some of the basics to debugging. In my videos, I get a lot of questions for help where people have errors and are not sure what the problem is. If they used some extremely simple debugging, they'd realize how obvious the answer is. Most of the time, the problem is a typo, followed closely by a misunderstanding of indentation and standards.
Standards how are how you organize your code. With python, unlike most languages, you define blocks of code, like functions, by indentation. Most python editors will automatically indent for you where it is necessary. With this, if you are ever coding along and find python automatically indenting you where you don't think it should, this should raise a flag for you to figure out.
There are some more in-depth common-issues that you'll find from time to time, you can find more debugging videos by searching for debugging in my channel. For now I will just keep these ones basic. The first error we'll discuss is the NameError: is not defined.
As obvious as this might appear to you, this gets people amazingly frequently. Just learn to recognize the "is not defined" as a high chance that you have typoed the definition of the variable or when you are calling it. Basically, you typo-ed somewhere.
variable = 55 print(varaible)Fail.
Next up, we have indentation issues. You will see "expected an indented block" as a popup when you never enter an indented block for something that requires it, like a function.
def task1(): def task2(): print('more tasks')
Nope.
So there was a lacking expected indent. How about an unexpected one?
def task(): print ('stuff') print('more stuff') print('stuff')
Denied.
Now what happens when you don't close off your strings and move to another line?
def task(): print('some people find themselves committing this too print('ouch...')
Nay. EOL while scanning string literal.
So there we have just a handful of very common errors. There are others from time to time, but they tend to be typos or indent issues quite often.