The Plain Text Is a Lie
There is no such thing as plain text
“But I see .txt files all the time” you say. “My source code is plain text” you claim. “What about web pages?!” you frantically ask. True, each of those things is comprised of text. The plain part is the problem. Plain denotes default or normal. There is no such thing. Computers store and transmit data in a number of methods; each are anything but plain. If you write software, design websites or test systems where even a single character of text is accepted as input, displayed as output, transmitted to another system or stored for later – please read on to learn why the plain text is a lie!
The topic of text handling applies to many disciplines:
- UX/web designers – Your UX is the last mile of displaying text to users.
- API developers – Your APIs should tell your consumers what languages, encodings and character sets your service supports.
- DBAs – You should know what kinds of text your database can handle.
- App developers – You apps should not crash when non-English characters are encountered.
After reading this article you will …
- … understand why text encodings are important.
- … have some best practices for handling text in your tool belt.
- … know a bit about how computers deal with text.
This topic has been extensively written about already. I highly recommend reading Joel Spolsky’s The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!). You should also read up on how your system handles strings. Then, go read how the APIs you talk to send/receive strings. Pythonistas, check out Ned Batchelder’s Pragmatic Unicode presentation.
OK, let’s get started!
Part I – Gallery of FAIL or “When text goes wrong, by audience”
Let’s start off by demonstrating how text handling can fail, and fail hard. The following screen shots and snippets show some of the ways text handling can fail and who should care about the type of failure.
UX and web people
The above image shows the English wikipedia article on Résumés with garbled text. Garbled text can happen if your web pages don’t specify an encoding or character set in your markup. Specifying the wrong encoding can also cause garbled text. XML and JavaScript need correct character sets too. It’s important to note that no error or exception was raised here. The text looks wrong to the user, but the failure happens silently.
This article on Tokyo above is displayed in a language (Aramaic) that my fonts don’t support. Instead of a symbol, we see a box with a number identifying the un-showable character. If you think that example is too contrived, here is a more commonly used symbol: a 16th note from sheet music. Many perfectly valid characters are not supported by widely used fonts. Specialized web fonts might not support the characters you need.
API developers
1 2 |
|
The result of this API call (example source) is similar to the last two examples: nonsense text. This can happen if the client and server use different text encodings. By the way, this situation happens so often that there’s a term for it: Mojibake.
Here are some client/server scenarios resulting in Mojibake:
- The server didn’t document their encoding and the client guessed the wrong encoding.
- The server or client inherit the encoding of their execution environment (virtual machine, OS, parent process, etc.), but the execution environment’s settings changed from their original values.
DBAs
Database systems can be misconfigured such that characters sent to the database are not stored accurately. In this example, the offending characters are replaced with the imaginatively-named Replacement Character (“�”). The original characters are forever lost. Worse still, replacement characters will be returned by your queries and ultimately shown to your users. Sometimes, offending characters will be omitted from the stored value or replaced with a nearest match supported character. In both scenarios the database has mangled the original data.
App developers
1 2 3 4 5 |
|
The top image shows the 500 page of an app that crashed when improperly encoding. In the Scala error message (bottom), a property file was read in ISO-8859-1 encoding but had UTF-8 encoded bytes in it. This caused the unit test to fail.
Your source code, web pages, properties files, and any other text artifact you work with has an encoding. Every tool in your development tool chain (local server, terminal, editor, browser, CI system, etc.) is a potential failure point if these encodings are not honoroed.
Part II – Avoid text handling problems
Ghost in the machine
You’ve seen examples of failure and (hopefully) are wondering how such failures can be avoided. To avoid failure you must ask yourself one question: “Can my system store and transmit a ghost?”
GHOST (code point U+1F47B) is a valid (albeit weird) part of the Unicode standard. Unicode is a system of storing and manipulating text that supports thousands of languages. Using Unicode properly will go a long way to prevent text handling problems. Thus, if your system can store, transmit, read and write GHOST then you’re doing it right. But how to handle this GHOST?
Some Terminology
You need to know some terms before the rest of this article will make any sense.
- Unicode object
- A datatype that lets you operate on Unicode text.
- Byte-string
- A sequence of bytes (octets).
- Encode
- To turn a Unicode object into a byte-string, where the bytes follow an encoding.
- Encoding (noun)
- A standard about what a byte means, like: When you see 01000001, it means “A”.
- Decode
- The inverse operation of encode. To turn a byte-string of a certain encoding into a Unicode object.
Remembering the difference between encode and decode can be difficult. One trick to keep them straight is to think of Unicode objects as the ideal state of being (thanks, Joel Spolksy) and byte-strings as strange, cryptic sequences.  Some systems use different terms but the ideas still apply. For example: Java Strings are Unicode objects and you can encode/decode to/from byte-strings with them.
Now that you’ve got the necessary terminology under your belt, let’s prevent text handling problems in our system by making a sandwich; a Unicode sandwich!
Make a Unicode sandwich
Analogy credit: Ned Batchelder coined the Unicode sandwich analogy in his Pragmatic Unicode presentation at PyCon 2012 (video). It’s so clever that I can’t resist re-using it in this article!
In this analogy the pieces of bread on the top and bottom are regions of your code where you deal with byte-strings. The meat in the middle is where your system deals in Unicode objects. The top bread is input into your system such as database query results, file reads or HTTP responses. The bottom bread is output from your system such as writing files or sending HTTP responses. The meat is your business logic.
Good sandwiches are meaty
Your goal is to keep the bread thin and the meat thick. You can achieve this by decoding from byte-strings to Unicode objects as early as you can; perhaps immediately after arrival from another system. Similarly, you should do your encoding from Unicode objects into byte-strings at the last possible moment, such as right before transmitting text to another system.
Working with Unicode inside your system gives you a common ground of text handling that will largely avoid the errors we’ve seen at the top of this article. If you don’t deal in Unicode inside your system then you are limiting the languages you support at best and exposing yourself to text handling bugs at worst!
The best sandwich bread is UTF-8
Your system ultimately needs to send and receive byte-strings at some point, so you must choose an encoding for your byte-strings. Encodings are not created equal! Some encodings only support one language. Some support only similar languages (for example, German and French but not Arabic). Never assume your system will only encounter languages you speak or write! Ideally you will choose encodings that support a great many languages.
UTF-8 is the best general purpose encoding for your byte-strings. You’ll learn why UTF-8 is an excellent encoding choice later in this article in the Unicode: One standard to rule them all section. For now I recommend you:
- Choose UTF-8 for all byte-strings.
- Configure your system to use this encoding explicitly. Do not rely on the parent system (OS, VM, etc.) to provide an encoding since system settings might change over time.
- Document your encoding choice in both public facing and internal documentation.
The UTF-8 encoding supports all the text you’d ever want. Yet, in this imperfect world you might be forced to use a more limited encoding such as ISO-8859-1 or Windows-1252 when interfacing with other systems. Working with a limited encoding presents problems when decoding to and encoding from Unicode: not every encoding supports the full Unicode range of characters. You must test how your system converts between your byte-strings and Unicode objects. In other words, test between the meat and the bread.
Testing between the meat and the bread
The critical areas to test are where bytes strings are decoded to Unicode objects and where Unicode objects are encoded into byte-strings. If you’ve followed the advice of this article thus far then the rest of your app logic should operate exclusively in Unicode objects. Here is a handy table of how to test regions of your system that encode and decode:
| Scenario | Test Strategy |
| My input encoding doesn’t support full Unicode. | Test that non-English† characters are faithfully decoded to Unicode. |
| My output encoding doesn’t support full Unicode. |
Test that supported non-English characters are faithfully encoded to byte-strings. Test that your system behaves correctly‡ when asked to encode un-supported characters. |
| My input(output) encoding supports full Unicode. | Test that non-English characters are faithfully decoded(decoded) to(from) Unicode. |
† English characters and Arabic numerals (0 - 9) are bad test cases because their byte values are identical across many encodings.
‡ Correctly is in the eye of the beholder. Some systems choose to raise an exception. Others choose to replace the offending character with a replacement character. Lastly, some systems simply omit the offending character. The choice is up to you, but they’re all terrible. Seriously, just use UTF-8.
Unicode sandwich applies to new projects and legacy systems
Using UTF-8 for I/O, Unicode inside and testing the in-between points will save you from pain and bugs. If you’re building a new system then you have the opportunity to design it with Unicode in mind. If you have an existing system, it is worth your time to audit how your system handles text.
With the practical stuff out of the way, let’s dive deeper into computers and text!
Part III – Encodings, Unicode and how computers handle text
We’ve talked about how you should use Unicode, encodings and byte-strings in your system to handle text. You may be wondering why text handling is so painful at times. Why are there so many encodings and why don’t they all work together in harmony? I’ll attempt to explain a bit of history behind text handling in computers. Understanding this history should shed some light on why text handling can be so painful.
To make things interesting, let’s pretend we are inventing how computers will handle text. Also assume we live in the United States and speak only English. That’s a pretty ignorant assumption for real world software development, but it simplifies our process.
ASCII: Works great (if you want to ignore most of the world)
Our challenge is to invent how computers handle text. Morse code is an encoding that pre-dates digital computers but provides a model for our approach: Each character has a transmission sequence of dots and dashes to represent it. We’ll need to make a few changes and additions though…
Rather than dots and dashes we can use 1’s and 0’s (binary). Let’s also use a consistent number of bits per character so that it’s easy to know when one character ends and another begins. To support US English we need to map a binary sequence to each of the following:
- a-z
- A-Z
- 0-9
- “ ”(space)
- !“#$%&‘()*+,–./:;<=>?@[\]^_`{|}~
- Control characters like “ring a bell”, “make a new line”, etc.
That’s 96 printable characters and some control characters for a total of 128 characters. 128 is 27, so we can send these characters in seven-bit sequences. Since computers use eight-bit bytes, let’s decide to send eight bits per character but ignore the last bit. We have just invented the ASCII encoding!
ASCII forms the root influence of many text encodings still used today. In fact, at one time ASCII was the law: U.S. President Lyndon B. Johnson mandated that all computers purchased by the United States federal government support ASCII in 1968.
International and OEM standards: Supporting other languages
Starting with similar languages to US English
We need more space to pack in more symbols if we want to support other languages and other symbols like currencies. It seems reasonable that people typically deal with a block of languages that are geographically or politically related, and when we’re lucky those languages share many of the same symbols. Given that assumption we can create several standards; each one for a block of languages!
For each block, we can keep the first 128 characters as-is from ASCII (identical bit sequences) so that the US English characters and Arabic numerals are still supported. We can then use the eighth bit for data instead of ignoring it. That would give us eight bits per character and a total of 256 characters to work with (double ASCII’s paltry 128). Now let’s apply that eight bit.
A bunch of countries in Western Europe use the same latin alphabet plus special diacritics (also known as accent marks) like ü or é or ß. In fact, we can pack enough extra characters in those last 128 slots to support 29 other languages like Afrikaans, German, Swahili and Icelandic. Our Western European language block encoding is ready! We call this type of encoding a single-byte encoding because every character is represented by exactly one byte.
Additional single byte encodings for other language blocks
We can repeat the same process we used to create our Western European language encoding to develop other single-byte encodings for other language blocks; each a 256 character set! To give one more example, let’s build a single byte coding for Arabic.
Again, we take the first 128 ASCII characters as-is, then fill up the last 128 with the Arabic alphabet We’ve got some space left over. Arabic has some diacritics as well, so let’s use some of the leftover slots to hold diacritic marks that are only valid when combined with other letters.
 Chinese, Japanese and Korean for example. That’s OK, we’ll just use multiple bytes per character to get more room. As you may have guessed, these encodings are called multibyte encodings. Sometimes we choose to use the same number of bytes for every character (fixed width multibyte encodings) and sometimes we might choose to use different byte lengths (variable width multibyte encodings) to save space.
Ratifying our encodings to standards
After we’ve built several of these encodings (Russian, Greek, Simplified Chinese, etc.) we can ratify them as international standards such as ISO-8859 for single byte encodings. We previously built ISO-8895-1 (Western European) and ISO-8859-6 (Latin/Arabic). International standards for multibyte encodings exist too. People who use the same standard can communicate without problems.
The international standards like ISO-8895 are only part of the story. Companies like Microsoft and IBM created their own standards (so-called OEM standards or code pages). Some OEM standards map to international standards, some almost-but-not-quite map (see Windows-1252) and some are completely different.
Our standards have problems
Our standards and code pages are better than ASCII but there are a number of problems remaining:
- How do we intermix different languages in the same document?
- What if our standards run out of room for new symbols?
- There is no rosetta stone to allow communication between systems that use different encodings.
Enter Unicode.
Unicode: One standard to rule them all
As mentioned earlier, Unicode is a single standard supporting thousands of languages. Unicode addresses the limitations of byte encodings by operating at a higher level than simple byte representations of characters. The foundation of Unicode is an über list of symbols chosen by a multinational committee.
Unicode keeps a gigantic numbered list of all the symbols of all the supported languages. The items in this list are called code points and are not concerned with bytes, how computers represent them, or what they look like on screen. They’re just numbered items, like:
a LATIN SMALL LETTER A – U+0061
東 Pinyin: dōng, Chaizi: shi,ba,ri – U+6771
☃ SNOWMAN – U+2603
We have virtually unlimited space to work with. The Unicode standards supports a maximum of 1,114,112 items. That is more than enough to express the world’s active written languages, some historical languages and miscellaneous symbols. Some of the slots are even undefined and left to the user to decide what they mean. These spaces have been used for wacky things like Klingon and Elvish.
Fun fact: Apple Inc. uses U+F8FF in the Private Use Area of Unicode for their logo symbol (). If you don’t see the Apple logo in parenthesis in the preceding sentence, then your system doesn’t agree with Apple’s use of U+F8FF.
OK, we have our gigantic list of code points. All we need to do is devise an encoding scheme to encode unicode objects (which now we know are lists of code points) into byte-strings for transmission over the wire to other systems.
UTF-8
UTF-8 encodes every Unicode code point in between one and four byte sequences. Here are some cool features of UTF-8:
- Popularity – It’s the dominant encoding of the world wide web since 2010.
- Simplicity – No need to transmit byte order information or worry about endianness in transmissions.
- Backwards compatibility – The first 128 byte sequences are identical to ASCII.
UCS-2: Old and busted
UCS-2 is a fixed width, two-byte encoding. In the mid-nineties, Unicode added code points that cannot be expressed in the two-byte system. Thus, UCS-2 is deprecated in favor of UTF-16.
- UCS-2 was the original Java String class’s internal representation
- C Python 2 and 3 use UCS-2 if compiled with default options
- Microsoft Windows OS API used UCS-2 prior to Windows 2000
UTF-16: UCS-2++
UTF-16 extends UCS-2 by adding support for the code points that can’t be expressed in a two-byte system. You can find UTF-16 in:
- Windows 2000 and later’s OS API
- The Java String class
- .NET environment
- OS X and iOS’s NSString type
UTF-32: Large bytes, simple representation
UTF-32 is a simple 1:1 mapping of code points to four-byte values. C Python uses UTF-32 for internal representation of Unicode if compiled with a certain flag.
Conclusion
We’ve seen how text handling can go wrong. We’ve learned how to design and test our systems with Unicode in mind. Finally, we’ve learned a bit of history of text encodings. There is a lot more to the topic of text, but for now I ask you do to the following:
- Examine your system to see if you’re using Unicode inside
- Use UTF-8 when reading and writing data
- Know that the plain text is a lie!
Thanks for reading!


Toggle navigation
angular-parse
Dashboard
Notification
Notification
{{currentUser.username}}
User profile
Settings
Help
Sign out


Google Earth Enterprise Documentation Home | Troubleshooting and error messages
Troubleshoot connection to GEE Server
If the Earth EC is unable to connect to your GEE Server, you'll see the error, "Google Earth failed to connect to database":
To resolve this issue, try these troubleshooting steps:
- Make sure the Server URL (
http://hostname/path
) is correct in the "Google Earth - Select Server" dialog box, which appears when you start EC:- The hostname should match the fully qualified domain name of your GEE Server.
- The path in the URL should match the Publish Point shown in the GEE Server UI. This path indicates the location where the database was published.
- Look for client EC requests in the GEE Server logs:
- Open a terminal window on the GEE Server.
- Type the command
tail -f /opt/google/gehttpd/logs/access_log
- Start EC and try connecting to your GEE Server.
In the terminal window, you should see time-stamped access requests in the logs from the client, starting with a GET request for thedbRoot
file, followed byflatfile
requests—for example:
<IP_ADDRESS> - - [08/Feb/2016:09:10:25 -0800] "GET /3d/dbRoot.v5?hl=en&gl=us&output=proto&cv=<IP_ADDRESS>7&ct=ec HTTP/1.1" 200 4512 "-" "GoogleEarth/<IP_ADDRESS>7(Windows;Microsoft Windows (6.2.9200.0);en;kml:2.2;client:EC;type:default)" <IP_ADDRESS> - - [08/Feb/2016:09:10:27 -0800] "GET /3d/dbRoot.v5?db=tm&hl=en&gl=us&output=proto&cv=<IP_ADDRESS>7&ct=ec HTTP/1.1" 200 1161 "-" "GoogleEarth/<IP_ADDRESS>7(Windows;Microsoft Windows (6.2.9200.0);en;kml:2.2;client:EC;type:default)"
<IP_ADDRESS> - - [08/Feb/2016:09:10:27 -0800] "GET /3d/flatfile?lf-0-icons/help_l.png&h=32 HTTP/1.1" 200 426 "-" "GoogleEarth/<IP_ADDRESS>7(Windows;Microsoft Windows (6.2.9200.0);en;kml:2.2;client:EC;type:default)"
If no new entries appear in the logs, then EC was unable to establish a connection to the GEE Server. This may be due to a network or firewall problem, as described in the next step.
- Verify network connectivity from EC to the GEE Server:
- The firewall on the host might be blocking outbound connections - Please check your firewall. Also, you can use tools such as Wireshark or Fiddler to capture HTTP traffic on your computer and confirm that EC is sending requests to the GEE Server.
- The firewall on the GEE Server might be blocking inbound connections from external hosts on port 80. Check whether firewalls like
iptables
are enabled on the server and are configured with packet-filtering rules. - Network or web proxies might be causing connection issues. Consult your network administrator.
- Use tools such as
ping
ortraceroute
on the client machine to diagnose connectivity issues.


Package com.google.protobuf
Interface DescriptorProtos.FieldOptionsOrBuilder
- All Superinterfaces:
com.google.protobuf.GeneratedMessageV3.ExtendableMessageOrBuilder<DescriptorProtos.FieldOptions>
,com.google.protobuf.MessageLiteOrBuilder
,com.google.protobuf.MessageOrBuilder
- All Known Implementing Classes:
DescriptorProtos.FieldOptions
,DescriptorProtos.FieldOptions.Builder
- Enclosing class:
- DescriptorProtos
public static interface DescriptorProtos.FieldOptionsOrBuilder
extends com.google.protobuf.GeneratedMessageV3.ExtendableMessageOrBuilder<DescriptorProtos.FieldOptions>
-
Method Summary
Modifier and TypeMethodDescriptiongetCtype()
The ctype option instructs the C++ code generator to use a different representation of the field than it normally would.boolean
Is this field deprecated? Depending on the target platform, this can emit Deprecated annotations for accessors, or it will be completely ignored; in the very least, this is a formalization for deprecating fields.The jstype option determines the JavaScript type used for values of the field.boolean
getLazy()
Should this field be parsed lazily? Lazy applies only to message-type fields.boolean
The packed option can be enabled for repeated primitive fields to enable a more efficient representation on the wire.getUninterpretedOption
(int index) The parser stores options it doesn't recognize here.int
The parser stores options it doesn't recognize here.The parser stores options it doesn't recognize here.getUninterpretedOptionOrBuilder
(int index) The parser stores options it doesn't recognize here.The parser stores options it doesn't recognize here.boolean
getWeak()
For Google-internal migration only.boolean
hasCtype()
The ctype option instructs the C++ code generator to use a different representation of the field than it normally would.boolean
Is this field deprecated? Depending on the target platform, this can emit Deprecated annotations for accessors, or it will be completely ignored; in the very least, this is a formalization for deprecating fields.boolean
The jstype option determines the JavaScript type used for values of the field.boolean
hasLazy()
Should this field be parsed lazily? Lazy applies only to message-type fields.boolean
The packed option can be enabled for repeated primitive fields to enable a more efficient representation on the wire.boolean
hasWeak()
For Google-internal migration only.Methods inherited from interface com.google.protobuf.GeneratedMessageV3.ExtendableMessageOrBuilder
getDefaultInstanceForType, getExtension, getExtension, getExtension, getExtension, getExtension, getExtension, getExtensionCount, getExtensionCount, getExtensionCount, hasExtension, hasExtension, hasExtension
Methods inherited from interface com.google.protobuf.MessageLiteOrBuilder
isInitialized
Methods inherited from interface com.google.protobuf.MessageOrBuilder
findInitializationErrors, getAllFields, getDescriptorForType, getField, getInitializationErrorString, getOneofFieldDescriptor, getRepeatedField, getRepeatedFieldCount, getUnknownFields, hasField, hasOneof
-
Method Details
-
hasCtype
boolean hasCtype()The ctype option instructs the C++ code generator to use a different representation of the field than it normally would. See the specific options below. This option is not yet implemented in the open source release -- sorry, we'll try to include it in a future version!
optional .google.protobuf.FieldOptions.CType ctype = 1 [default = STRING];
- Returns:
- Whether the ctype field is set.
-
getCtype
DescriptorProtos.FieldOptions.CType getCtype()The ctype option instructs the C++ code generator to use a different representation of the field than it normally would. See the specific options below. This option is not yet implemented in the open source release -- sorry, we'll try to include it in a future version!
optional .google.protobuf.FieldOptions.CType ctype = 1 [default = STRING];
- Returns:
- The ctype.
-
hasPacked
boolean hasPacked()The packed option can be enabled for repeated primitive fields to enable a more efficient representation on the wire. Rather than repeatedly writing the tag and type for each element, the entire array is encoded as a single length-delimited blob. In proto3, only explicit setting it to false will avoid using packed encoding.
optional bool packed = 2;
- Returns:
- Whether the packed field is set.
-
getPacked
boolean getPacked()The packed option can be enabled for repeated primitive fields to enable a more efficient representation on the wire. Rather than repeatedly writing the tag and type for each element, the entire array is encoded as a single length-delimited blob. In proto3, only explicit setting it to false will avoid using packed encoding.
optional bool packed = 2;
- Returns:
- The packed.
-
hasJstype
boolean hasJstype()The jstype option determines the JavaScript type used for values of the field. The option is permitted only for 64 bit integral and fixed types (int64, uint64, sint64, fixed64, sfixed64). A field with jstype JS_STRING is represented as JavaScript string, which avoids loss of precision that can happen when a large value is converted to a floating point JavaScript. Specifying JS_NUMBER for the jstype causes the generated JavaScript code to use the JavaScript "number" type. The behavior of the default option JS_NORMAL is implementation dependent. This option is an enum to permit additional types to be added, e.g. goog.math.Integer.
optional .google.protobuf.FieldOptions.JSType jstype = 6 [default = JS_NORMAL];
- Returns:
- Whether the jstype field is set.
-
getJstype
DescriptorProtos.FieldOptions.JSType getJstype()The jstype option determines the JavaScript type used for values of the field. The option is permitted only for 64 bit integral and fixed types (int64, uint64, sint64, fixed64, sfixed64). A field with jstype JS_STRING is represented as JavaScript string, which avoids loss of precision that can happen when a large value is converted to a floating point JavaScript. Specifying JS_NUMBER for the jstype causes the generated JavaScript code to use the JavaScript "number" type. The behavior of the default option JS_NORMAL is implementation dependent. This option is an enum to permit additional types to be added, e.g. goog.math.Integer.
optional .google.protobuf.FieldOptions.JSType jstype = 6 [default = JS_NORMAL];
- Returns:
- The jstype.
-
hasLazy
boolean hasLazy()Should this field be parsed lazily? Lazy applies only to message-type fields. It means that when the outer message is initially parsed, the inner message's contents will not be parsed but instead stored in encoded form. The inner message will actually be parsed when it is first accessed. This is only a hint. Implementations are free to choose whether to use eager or lazy parsing regardless of the value of this option. However, setting this option true suggests that the protocol author believes that using lazy parsing on this field is worth the additional bookkeeping overhead typically needed to implement it. This option does not affect the public interface of any generated code; all method signatures remain the same. Furthermore, thread-safety of the interface is not affected by this option; const methods remain safe to call from multiple threads concurrently, while non-const methods continue to require exclusive access. Note that implementations may choose not to check required fields within a lazy sub-message. That is, calling IsInitialized() on the outer message may return true even if the inner message has missing required fields. This is necessary because otherwise the inner message would have to be parsed in order to perform the check, defeating the purpose of lazy parsing. An implementation which chooses not to check required fields must be consistent about it. That is, for any particular sub-message, the implementation must either *always* check its required fields, or *never* check its required fields, regardless of whether or not the message has been parsed.
optional bool lazy = 5 [default = false];
- Returns:
- Whether the lazy field is set.
-
getLazy
boolean getLazy()Should this field be parsed lazily? Lazy applies only to message-type fields. It means that when the outer message is initially parsed, the inner message's contents will not be parsed but instead stored in encoded form. The inner message will actually be parsed when it is first accessed. This is only a hint. Implementations are free to choose whether to use eager or lazy parsing regardless of the value of this option. However, setting this option true suggests that the protocol author believes that using lazy parsing on this field is worth the additional bookkeeping overhead typically needed to implement it. This option does not affect the public interface of any generated code; all method signatures remain the same. Furthermore, thread-safety of the interface is not affected by this option; const methods remain safe to call from multiple threads concurrently, while non-const methods continue to require exclusive access. Note that implementations may choose not to check required fields within a lazy sub-message. That is, calling IsInitialized() on the outer message may return true even if the inner message has missing required fields. This is necessary because otherwise the inner message would have to be parsed in order to perform the check, defeating the purpose of lazy parsing. An implementation which chooses not to check required fields must be consistent about it. That is, for any particular sub-message, the implementation must either *always* check its required fields, or *never* check its required fields, regardless of whether or not the message has been parsed.
optional bool lazy = 5 [default = false];
- Returns:
- The lazy.
-
hasDeprecated
boolean hasDeprecated()Is this field deprecated? Depending on the target platform, this can emit Deprecated annotations for accessors, or it will be completely ignored; in the very least, this is a formalization for deprecating fields.
optional bool deprecated = 3 [default = false];
- Returns:
- Whether the deprecated field is set.
-
getDeprecated
boolean getDeprecated()Is this field deprecated? Depending on the target platform, this can emit Deprecated annotations for accessors, or it will be completely ignored; in the very least, this is a formalization for deprecating fields.
optional bool deprecated = 3 [default = false];
- Returns:
- The deprecated.
-
hasWeak
boolean hasWeak()For Google-internal migration only. Do not use.
optional bool weak = 10 [default = false];
- Returns:
- Whether the weak field is set.
-
getWeak
boolean getWeak()For Google-internal migration only. Do not use.
optional bool weak = 10 [default = false];
- Returns:
- The weak.
-
getUninterpretedOptionList
List<DescriptorProtos.UninterpretedOption> getUninterpretedOptionList()The parser stores options it doesn't recognize here. See above.
repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999;
-
getUninterpretedOption
The parser stores options it doesn't recognize here. See above.
repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999;
-
getUninterpretedOptionCount
int getUninterpretedOptionCount()The parser stores options it doesn't recognize here. See above.
repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999;
-
getUninterpretedOptionOrBuilderList
List<? extends DescriptorProtos.UninterpretedOptionOrBuilder> getUninterpretedOptionOrBuilderList()The parser stores options it doesn't recognize here. See above.
repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999;
-
getUninterpretedOptionOrBuilder
The parser stores options it doesn't recognize here. See above.
repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999;
-


--- title: Could You Modify It ‘To Stop Students From Becoming This Advanced?’ layout: post ---
Khan’s programmer, Ben Kamens, has heard from teachers who’ve seen Khan Academy presentations and loved the idea but wondered whether they could modify it “to stop students from becoming this advanced.”


---
layout: layout
---
{% include work.html %}
{% include education.html %}
{% include publications.html %}
{% include posters.html %}
{% include research.html %}
{% include workshops.html %}
{% include initiatives.html %}
{% include development.html %}
{% include skills.html %}
{% include interests.html %}


197 Red
194 Blue
136 Green
121 Fish
97 Cyan
93 Yellow
87 Rainbow
79 Pink
78 Purple
71 Orange
66 Lime
59 White
45 Salmon
43 Black
41 Myrtle
40 Puce
35 Teal
33 Aqua
32 Azure
29 Brown
28 Peach
26 red
40 +1618
18 +1617
16 +1802
16 +1603
12 +1646
6 +1917
6 +1802
5 +1603
4 +1541
4 +1207
3 +1774
1 +1978
1 +1702
1 +1508
with
Free free to get in touch to riff on ideas and possibilities! I'd love to hear what want to make, and am happy to help think of ways to make it more public.


|
Formula1 Racing Simulator
|
#include <Circuit.h>
Public Attributes | |
| int | lap = 50 |
| Number of laps in the race. More... | |
| float | Circuit_length = 5371.0 |
| Circuit length in meter. More... | |
| float | right_bends = 4.0 |
| Number of right bends in the circuit. More... | |
| float | left_bends = 5.0 |
| Number of left bends in the circuit. More... | |
| float Circuit::Circuit_length = 5371.0 |
Circuit length in meter.
| int Circuit::lap = 50 |
Number of laps in the race.
| float Circuit::left_bends = 5.0 |
Number of left bends in the circuit.
| float Circuit::right_bends = 4.0 |
Number of right bends in the circuit.


{% extends "base.html" %} {% block css %} {% endblock %} {% block main %} Your account has been activated. {% endblock %}


 I went to school. I ran cross country.I played guitar.I hung out with my friends. I aspired to become a trauma surgeon. I ate three meals a day. I slept at night. But those things I just described to you are hard for me to remember now. Why? Because for the past 3 years of my life I have, without realizing it, given my life over to 'Addy.'
Before Addy, my drug habits consisted mainly of alcohol (on weekends and in very moderate amounts), weed (maybe once a week), and on very rare occasions, coke and/or smack (but never to the point of addiction).  When my pediatrician suggested that I switch from Ritalin to Adderall, or as he called it, 'The newest, longest-acting and most effective ADD medication', I was a little nervous because after years of taking Ritalin I had grown quite accustomed to it and was reluctant to part w / it. I had heard of people abusing Ritalin before but never had a desire to try it myself. I liked the way it made me feel when I took it as prescribed. I told my doctor that I would agree to meet this 'Addy' on a trial basis. I was 15 years old at that time.
So, at that point, I allowed Addy into my life in a 10 mg dose at breakfast and a second, 5 mg dose at noon. My life changed radically in the months that followed. I noticed that about an hour after taking my morning dose, time turned into a somewhat abstract thing. An activity that would usually be boring and dull, became the most exciting thing in the world with Addy there to keep me company. All of my senses were amplified. I didn't have to bother with trivial things like eating and sleeping. I was intensely FOCUSED on everything around me and inside of me. It felt so good to be in my skin. To sum it up in a word : Supernatural. I felt like a goddess when I was with Addy. Three years flew by, before I even had the chance to blink my dilated eyes. Life was perfect, or so I thought.
Let me take this opportunity to explain some things that I have discovered the hard way about Miss Addy. Addy is sly : She can trick me into believing that everything is OK in my life even if, in reality, things are falling apart.  Addy is possessive : She will only accept first place in my life and nothing can come before her.  Addy is beautiful but costly : The more I indulge in her, the more I HAVE to be with her just to function. Addy is powerful : She has become the impetus behind my every action.
She has created a burning desire for herself in my body and mind. Addy promised me the moon but then turned right around and stole it from the sky. This is real.   I graduated Valedictorian of my class. I had a lot of friends. I was awarded a $24,000 scholarship to college. But that was then........
Now:
It's 8:51 on a Sunday morning and I have watched the sun come up again for the third day in a row. I haven't eaten substantial food in 4 days and I haven't slept at all in 3 days. I jump everytime I hear the slightest sound. I am chewing gum and my jaws are moving as fast as my brain.  I have eaten/snorted 250 mgs of Addy in the past 3 days. My heart is beating really hard and fast. I see lights flashing but it is completely dark in this room except for the computer monitor. Two years ago this type of come-down would've scared me to death, but I know this feeling so well that I don't even really notice it anymore. Addy is calling me...................I have to do more so that I won't crash............. Here we go again...........
Experience Reports are the writings and opinions of the individual authors who submit them.
Some of the activities described are dangerous and/or illegal and none are recommended by Erowid.


The example in this section uses an intracommunicator.
Example
This example uses a user-defined operation to produce a segmented scan. A segmented scan takes, as input, a set of values and a set of logicals, and the logicals delineate the various segments of the scan. For example:
The operator that produces this effect is,
where,
Note that this is a non-commutative operator. C code that implements it is given below.
typedef struct { double val; int log; } SegScanPair; /* the user-defined function */ void segScan( SegScanPair *in, SegScanPair *inout, int *len, MPI_Datatype *dptr ) { int i; SegScanPair c; for (i=0; i< *len; ++i) { if ( in->log == inout->log ) c.val = in->val + inout->val; else c.val = inout->val; c.log = inout->log; *inout = c; in++; inout++; } }Note that the inout argument to the user-defined function corresponds to the right-hand operand of the operator. When using this operator, we must be careful to specify that it is non-commutative, as in the following.
int i,base; SeqScanPair a, answer; MPI_Op myOp; MPI_Datatype type[2] = {MPI_DOUBLE, MPI_INT}; MPI_Aint disp[2]; int blocklen[2] = { 1, 1}; MPI_Datatype sspair; /* explain to MPI how type SegScanPair is defined */ MPI_Address( a, disp); MPI_Address( a.log, disp+1); base = disp[0]; for (i=0; i<2; ++i) disp[i] -= base; MPI_Type_struct( 2, blocklen, disp, type, &sspair ); MPI_Type_commit( &sspair ); /* create the segmented-scan user-op */ MPI_Op_create( segScan, 0, &myOp ); ... MPI_Scan( a, answer, 1, sspair, myOp, comm );


Acceptance testing your Ember app typically involves verifying some user behavior. For example, you may want to test that the user can view the photos on your app’s index route.
Many of these tests rely on a given server state. In other words, you want to test that the user can view ten photos, given that ten photo resources exist on the server at the time the user boots up the app.
Once you’ve defined your routes, use the server
variable to define the initial server state directly in your tests:
test('I can view the photos', assert => {
server.createList('photo', 10);
visit('/');
andThen(function() {
assert.equal( find('img').length, 10 );
});
});
server.createList
uses the photo
factory to generate 10 database records. This way, Mirage’s database is populated when the Ember app boots and makes an AJAX request to fetch the photos data.
Mirage’s server will be reset after each test, so each test starts out with a clean database state.
The purpose of factories is to put code that’s relevant to a test as close to that test as possible. In the example above, we wanted to verify that the user would see ten photos, given those photos existed on the server. So, the server.createList('photo', 10)
call was directly in the test.
Say we wanted to test that when the user visits a details route for a photo titled “Sunset over Hyrule,” they would see that title in an <h1>
tag. One way to accomplish this would be to update the photo factory itself:
// mirage/factories/photo.js
import Mirage from 'ember-cli-mirage';
export default Mirage.Factory.extend({
title: 'Sunset over Hyrule'
});
The problem with this approach is that the desired change is very specific to this test. Suppose another test wanted to verify photos with different titles were displayed. Changing the factory to suit that case would break this test.
For this reason, create
and createList
allow you to override specific attributes that your factory has defined. This lets us keep relevant code near our tests, without making them brittle.
To override attributes, simply pass in an object as the last argument to create
or createList
with the attributes you want to override. Here’s what this may look like for the photos example.
First, let’s make our factory more generic:
// mirage/factories/photo.js
import Mirage from 'ember-cli-mirage';
export default Mirage.Factory.extend({
title(i) {
// Photo 1, Photo 2 etc.
return `Photo ${i}`;
}
});
Now, we can write our tests, overriding the factory-generated attributes where appropriate:
test("I can view the photos", assert => {
server.createList('photo', 10);
visit('/');
andThen(() => {
assert.equal( find('img').length, 10 );
});
});
test("I see the photo's title on a detail route", assert => {
let photo = server.create('photo', {title: 'Sunset over Hyrule'});
visit('/' + photo.id);
andThen(() => {
assert.equal( find('h1:contains(Sunset over Hyrule)').length, 1 );
});
});
We override title
in the second test since it’s relevant there, but we stick with the factory-generated defaults for the first test.
Often you’ll write tests against your application’s UI, which will verify that the proper data from Mirage was returned. However, because Mirage gives you a full client-side server, you can gain more confidence from your tests by asserting against Mirage’s server state, in addition to testing your Ember app’s UI.
There are two general approaches to this. First, you can assert directly against Mirage’s database:
test("I can change the lesson's title", assert => {
server.create('lesson', {title: 'My First Lesson'})
visit('/');
click('.Edit')
fillIn('input', 'Updated lesson');
click('.Save');
andThen(() => {
// Assert against our app's UI
assert.equal( find('h1:contains(Updated lesson)').length, 1 );
// Also check that the data was "persisted" to our backend
assert.equal( server.db.lessons[0].title, 'Updated lesson');
});
});
The next strategy is to temporarily override the server route that’s relevant to your test, and assert against the actual request that your Ember app sent:
test("I can change the lesson's title", function(assert) {
assert.expect(1);
let done = assert.async();
server.create('lesson', {title: 'My First Lesson'})
server.put('/lessons/:id', (schema, request) => {
let params = JSON.parse(request.requestBody);
// Here, we're asserting the params Mirage received are in the format you expect
assert.deepEqual(params, {...});
done();
});
visit('/');
click('.Edit')
fillIn('input', 'Updated lesson');
click('.Save');
});
Note that here, we’re overwriting any route handler you may defined for PUT to /lessons/:id
in your config.js
file, but only for this test. After this test, your Mirage server will be reset, and all the routes from config.js
will be used.
To test how your Ember app responds to a server error, overwrite a route handler within a test:
test('the user sees an error if the save attempt fails', function(assert) {
server.post('/questions', {errors: ['There was an error']}, 500);
visit('/');
click('.new');
fillIn('input', 'New question');
click('.save');
andThen(() => {
assert.equals(find('p:contains(There was an error)').length, 1);
});
});
This route handler definition is only in effect for the duration of this test, so as soon as it’s over any handler you have defined for POST to /questions
in your config.js
file will be used again.
You should now know enough to fake your production API and test your app using Mirage!


import json
import logging
from io import StringIO
from pathlib import Path
from .loaders.csv import CsvLoader
from .loaders.excel import ExcelLoader
from .loaders.json import JsonLoader
from .loaders.text import TextLoader
from .loaders.toml import TomlLoader
from .loaders.xml import XmlLoader
logger = logging.getLogger(__name__)
DEFAULT_LOADERS = [
CsvLoader,
ExcelLoader,
JsonLoader,
TextLoader,
TomlLoader,
XmlLoader
]
# FIXME: this can be done more elegant
def add_loader(loaders, loader):
extensions = loader.EXTENSIONS
logger.debug(f"Adding loader {loader} for extensions {extensions}")
for extension in extensions:
if extension in loaders:
raise LoaderError(f"There is already a loader for extension '{extension}'")
else:
loaders[extension] = loader
return loaders
class KneadException(Exception):
pass
class LoaderError(Exception):
pass
[docs]class Knead:
_data = None
_loaders = {}
for loader in DEFAULT_LOADERS:
add_loader(_loaders, loader)
[docs] def __init__(self, inp, parse_as = None, read_as = None, is_data = False, **kwargs):
"""
Creates a new ``Knead`` instance.
By default it is assumed that ``inp`` is a path to a file if it's a string.
The file extension will be used to determine the correct loader (e.g. ``.csv``
will get the :py:class:`CsvLoader <dataknead.csv.CsvLoader>`).
::
Knead("cities.csv")
If ``inp`` is not a string it is assumed to be a data structure (e.g. a list or dict).
::
Knead([1, 2, 3])
``parse_as`` is used when you're feeding ``Knead`` a string that is not a file
path, like the results of a HTTP call. In that case you need to specify the
loader yourself as a file extension (e.g. ``csv``, ``json``, etc.)
::
json = "[1,2,3]"
Knead(json, parse_as = "json")
``read_as`` is used when you're giving a path to a file that doesn't have a
valid extension, and doesn't have an extension at all. The same mechanism as ``parse_as``
is at work here: give it a file extension.
::
Knead("query", read_as = "csv")
``is_data`` is used if you want to feed ``Knead`` a string as data, and it shouldn't
be parsed at all
::
Knead("a line of text", is_data = True)
Any additional parameters will be passed on to the loader
::
Knead("cities.csv", has_header = True)
"""
logger.debug(f"Input: {inp}")
if parse_as:
# Process string like file
if not isinstance(inp, str):
raise TypeError(f"Input needs to be string, not {type(inp).__name__}")
loader = self._get_loader(parse_as)
f = StringIO(inp)
self._data = loader.read(f, **kwargs)
elif isinstance(inp, str) and not is_data:
# Either a path or a stringified data file
if not read_as:
read_as = Path(inp).suffix[1:]
loader = self._get_loader(read_as)
with open(inp) as f:
self._data = loader.read(f, **kwargs)
else:
# We assume this is parsed data, assign it
self._data = inp
def __repr__(self):
return json.dumps(self.data(), indent = 4)
def __str__(self):
return json.dumps(self.data(), indent = 4)
def _get_loader(self, extension):
logger.debug(f"Trying to find loader for extension '{extension}'")
if extension in self._loaders:
logger.debug(f"Found loader for '{extension}'")
return self._loaders[extension]
else:
raise KneadException(f"Could not find loader for extension '{extension}'")
[docs] def add_loader(self, loader):
"""
Adds a custom loader. See :ref:`extending dataknead <extending>`.
"""
add_loader(self._loaders, loader)
[docs] def apply(self, fn):
"""
This applies a function to the loaded data.
"""
self._data = fn(self.data())
return self
[docs] def data(self, check_instance = None):
"""
Returns the loaded data.
::
Knead("cities.csv").data() # Will return the parsed contents of cities.csv
To check if the data is in a specific type use ``check_instance``,
which will raise an exception if it's not correct
::
Knead([1,2,3]).data(check_instance = dict) # Will raise an error, it's a list
"""
datatype = type(self._data)
if check_instance and not isinstance(self._data, check_instance):
dtype = datatype.__name__
preftype = check_instance.__name__
raise Exception(f"Data of type {dtype} can not be processed, needs to be {preftype}")
else:
return self._data
[docs] def filter(self, fn):
"""
Applies a function on every item and removes it if it returns ``False``.
This only works when your data is a ``list``.
::
Knead([1,10,100]).filter(lambda v: v > 9).data() # Returns 10 and 100
"""
data = [row for row in self.data(check_instance = list) if fn(row)]
return Knead(data)
[docs] def keys(self):
"""
Returns the keys of the internal data
"""
return Knead(list(self.data().keys()))
[docs] def map(self, iteratee):
"""
Applies a function on every item and replaces it with the return value.
::
Knead([1, 4, 9]).map(lambda x:x * x).data() # [1, 16, 81]
If you have a list with dicts, you can give a string to return the
values of a specific key
::
Knead("cities.csv").map("city").write("city-names.csv")
# Is the same as
Knead("cities.csv").map(lambda c:c["city"]).write("city-names.csv")
To return multiple keys with values, you can use a tuple:
::
Knead("cities.csv").map(("city", "country")).write("city-country-names.csv")
# Is the same as
Knead("cities.csv").map(lambda c:{ "city" : c["city"], "country" : c["country"] }).write("city-country-names.csv")
# Or
def mapcity(city):
return {
"city" : city["city"],
"country" : city["country"]
}
Knead("cities.csv").map(mapcity).write("city-country-names.csv")
"""
data = self.data(check_instance = list)
# If 'iteratee' is a function, map over the data
if callable(iteratee):
data = [iteratee(row) for row in data]
# Shortcut, like 'pluck'
elif isinstance(iteratee, str):
data = [row[iteratee] for row in data]
# Another shortcut, for multiple keys
elif isinstance(iteratee, tuple):
data = [ { key:row[key] for key in iteratee } for row in data ]
else:
raise TypeError("Iteratee should be of type dict or function")
return Knead(data)
[docs] def values(self):
"""
Returns the values of the data
"""
return Knead(list(self.data().values()))
[docs] def write(self, path, write_as = None, **kwargs):
"""
Writes the data to a file. Type is implied by file extension.
::
Knead("cities.csv").write("cities.json")
To force the type to something else, pass the format to `write_as`.
::
Knead("cities.csv").map("city").write("cities.txt", write_as="csv")
Any additional parameters will be passed on to the loader:
::
Knead("cities.csv").write("cities.json", indent = 4)
Knead("cities.csv").map("city").write("ciites.csv", fieldnames=["city"])
"""
logger.debug(f"write(): {path}")
if not write_as:
write_as = Path(path).suffix[1:]
loader = self._get_loader(write_as)
with open(path, "w") as f:
loader.write(f, self.data(), **kwargs)
logger.debug(f"Wrote the data to {path}")


Working on it. Please wait ...
Search for Veteran
Please fill out first name, last name, and SSN. Middle name optional:
First Name:
Middle Name:
Last Name:
SSN:
Retrieve Veteran by Member ID
Member ID:
The search requires some additional information. Please follow the instructions above to execute your search.
Search Results


I'm a passionate full stack developer with Two year's of working experience with both server side and client side. My development stack of choice includes Angular 2/4/5, IONIC 2/5 , NodeJS and Java. I'm a firm believer in test driven development, continous integration and code review, and always strive to write beautiful and elegant code that solves complex problems easily.
Developed the Mobile Application for blind people. that take the picture and identify the objects in the image.
_Built with Ionic, Google Vision API, Watson Speech to text API.
Developed Online IDE for the event Progeeks in LOGIN 2015. including Gowtham Shanmugam to develop a beautifully usable product that allows users. to program according to provided specifications..
_Built with PHP ,MySQL
Coda Global: 18/05/2016-present
Worked on developing web and Mobile application for clients using technologies such as IONIC 2, Angular 2, NodeJs.
YuMe: 21/12/2015-30/04/2016
Major Project Internship - Worked on Automation of YFA and YFP using technologies such as Java, selenium and TestNG
Aruba NetWork: 02/06/2015-30/10/2015
Summer Internship - monitoring/handling the Access Points within the group level.sing technologies such Python, Pexpect and fdpexpect
Freelance:
Worked with a number of clients to provide their small businesses with a professional online presence.
Master of Computer Application (2013 - 2016) PSG College of Technology, Coimbatore CGPA 7.54


Enter in the
match patterns
to refresh (regular URLs will not work). One pattern per line.
Extensions
Save


>
This is a website that checks which type of a triangle you inqured and returns the result.
The three types of triangle you can come up with are:
If the sum of the lengths of any two sides of a triangle is less than or equal to the length of the third side, this CANNOT be a triangle hence NOT a triangle will be returned
Key in the length
Key in the height
Key in the hypotenuse


Something's gone terribly wrong! The monkeys have been dispatched to resolve it as soon as possible.
{% extends "layouts/main.html" %} {% block content_page %}
Something's gone terribly wrong! The monkeys have been dispatched to resolve it as soon as possible.


@* * Copyright 2017 HM Revenue & Customs * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. *@ @(fieldName: String, questionText: String, helpText: Option[String] = None)(implicit messages: Messages)


RAD Events can help you FIND events. ATTEND these events and you EARN points towards a RAD Event at the end of each quarter!
Learn More Find EventsNeed more people to attend your event? Want to host your event on RAD Events? Contact us and we can help you integrate your event with our forum.
123-456-6789


P
H
P
K
Error
Forbidden
Your client does not have permission to get this directory from this server.


News Feed
Marvin
20 followers
25
Refreshing and tasty! Would recommend for a healthy meal
Habitual Fix
10 Right Street
Rachel
5 followers
10
Reasonable price for a quick and healthy meal!
Habitual Fix
10 Right Street