Query Profiles
A profile is a summary of metrics collected for each query that Drill executes. Query profiles provide information that you can use to monitor and analyze query performance. When Drill executes a query, Drill writes the profile of each query to disk, which is either the local filesystem or a distributed file system, such as HDFS. As of Drill 1.11, Drill can store profiles in memory instead of writing them to disk. You can view query profiles in the Drill Web UI at http://<IP address or host name>:8047
. Starting in Drill 1.16, the Drill Web UI displays warning messages in the query profile if there are issues that have performance impact. In addition to viewing query profiles, you can modify, resubmit, or cancel queries from the Drill Web UI.
The Drill Web UI provides aggregate statistics across profile lists. Profile lists consist of data from major and minor fragments, operators, and input streams. Use query profiles in conjunction with Drill logs to debug issues.
Viewing a Query Profile
You can view query profiles in the Profiles tab of the Drill Web UI. When you select the Profiles tab, you see a list of queries than ran or are currently running in the cluster.
Starting in Drill 1.16, you can sort the columns on the Profiles page. For example, if you want to quickly locate queries with the longest running times, you can sort the Duration column in descending order such that the longest running queries appear at the top of the list. You can also use the Search Profiles field to filter the list of query profiles and then use the sort feature.
For example, the following image shows the query profiles filtered by the mapr user on a specific date with the Duration column sorted in descending order:
To see the profile for a specific query, click on the query in the profiles list.
The Query Profile section summarizes a few key details about the query, including:
- The state of the query, either running, completed, or failed.
- The node operating as the Foreman; the Drillbit that receives a query from the client or application becomes the Foreman and drives the entire query.
- The total number of minor fragments required to execute the query.
When you select a profile, notice that the URL in the address bar contains the QueryID, as shown in the following URL:
http://<drill_node>:8047/profiles/2aa98add-15b3-e155-5669-603c03bfde86
Query, Fragment, and Operator Identifiers
Metrics in a query profile are associated with a coordinate system of identifiers. Drill uses a coordinate system comprised of query, fragment, and operator identifiers to track query execution activities and resources. Drill assigns a unique identifier, the QueryID, to each query received and then assigns an identifier to each fragment and operator that executes the query. An example of a QueryID is 2aa98add-15b3-e155-5669-603c03bfde86. The following image shows an example of fragment and operator identifiers:
Query Profile Warnings
When Drill executes a query, the workload should be uniformly distributed across the fragments and operators processing the data. When you evaluate a query profile in the Drill Web UI, seeing a disproportionate distribution of work across fragments (in terms of time) or excess memory use typically indicates performance issues and requires some performance tuning.
Starting in Drill 1.16, the Drill Web UI displays warning messages in the query profile for the following issues:
- No query progress within a certain amount of time
- Operators spilling data to disk (operators do not have enough memory to complete operations completely in-memory)
- Operators spending significantly more time waiting for data than processing it
In addition to the warnings, the Operator Profiles section of the query profile also displays an icon in the column to indicate which operator is askew.
The following table lists the warnings, icons, configurable options related to the warnings, and option descriptions:
| Warning Message | Icon | Related Option(s) | Description |
|---|---|---|---|
| !WARNING: No fragments have made any progress in the last 300 seconds. (See Last Progress below.) | None | drill.exec.http.profile.warning.progress.threshold | If none of the fragments make any progress within the set time, a warning displays. Time is set in seconds. Default is 300 seconds (5 minutes). You can see fragment activity in the Fragment Profiles section of the query profile. |
| !WARNING: Some operators have data spilled to disk. This will result in performance loss. (See Avg Peak memory and Max Peak Memory below) | Look for the icon in the Avg Peak Memory and Max Peak Memory columns to find the operators that spilled data to disk. | See Sort-Based and Hash-Based Memory Constrained Operators for related options. | Hovering the mouse over the icon reveals the average number of spills. The higher the value, the greater the degradation |
| !WARNING: Some of the operators spent more time waiting for data than processing it. (See AVG WAIT TIME as compared to Average Process Time for the operators below) | Look for the icon in the Max Process Time or Max Wait Time columns to locate the operators that waited or processed too long. | drill.exec.http.profile.warning.time.skew.min | Sets the minimum threshold for operators with the longest processing or waiting fragment. When the slowest fragment hits this threshold and runs or waits at least 2X (default setting) longer than the average fragment, the Drill Web UI displays a warning message. Default is 2. The drill.exec.http.profile.warning.time.skew.ratio.process option sets the threshold ratio for processing. The drill.exec.http.profile.warning.time.skew.ratio.wait option sets the threshold ratio for waiting. |
| See warning for drill.exec.http.profile.warning.time.skew.min | drill.exec.http.profile.warning.time.skew.ratio.process | Defines the threshold ratio for processing. When the maxProcessing:avgProcessing ratio exceeds the defined threshold, the Drill Web UI displays a skew warning. Default is 2. | |
| See warning for drill.exec.http.profile.warning.time.skew.min | drill.exec.http.profile.warning.time.skew.ratio.wait | Defines the threshold ratio for waiting. When the maxWait:avgWait ratio exceeds the defined threshold, the Drill Web UI displays a skew warning. Default is 2. | |
| !WARNING: Some of the SCAN operators spent more time waiting for data than processing it. (See AVG WAIT TIME as compared to Average Process Time for the scan operators below) | Look for the icon in the Avg Wait Time or Avg Process Time columns to locate the scan operators that waited too long. | drill.exec.http.profile.warning.scan.wait.min | Sets a minimum threshold ratio for the scan operator to wait. When the average wait time exceeds the processing time the Drill Web UI displays the warning. Default is 60 seconds. |
Configuring Warning Threshold Options
You can configure thresholds for the warnings through configuration options in the drill-override.conf file, as shown:
http: {
profile.warning: {
progress.threshold: 300,
time.skew: {
min: 2,
ratio: {
process: 2
wait: 2
}
},
scan.wait.min: 60
},
...
}
You must restart Drillbits after you modify drill-override.conf.
The following sections describe the Fragment Profiles and Operator Profiles sections of the Drill Web UI in more detail.
Fragment Profiles
Fragment profiles provides an overview table and a major fragment block for each major fragment. Each row in the Overview table provides the number of minor fragments that Drill parallelized from each major fragment, as well as aggregate time and memory metrics for the minor fragments.
When you look at the fragment profiles, you may notice that some major fragments were parallelized into substantially fewer minor fragments, but happen to have the highest runtime. Or, you may notice certain minor fragments have a higher peak memory than others. When you notice these variations in execution, you can delve deeper into the profile by looking at the major fragment blocks.
Major fragment blocks correspond to a row in the Overview table. You can expand the blocks to see metrics for all of the minor fragments that were parallelized from each major fragment, including the host on which each minor fragment ran. Each row in the major fragment table presents the fragment state, time metrics, memory metrics, and aggregate input metrics of each minor fragment.
When looking at the minor fragment metrics, verify the state of the fragment. A fragment can have a “failed” state which may indicate an issue on the host. If the query itself fails, an operator may have run out of memory. If fragments running on a particular node are under performing, there may be multi-tenancy issues that you can address.
A graph illustrates the activity of major and minor fragments for the duration of the query. The graph correlates with the visualized plan graph in the Visualized Plan tab. Each color in the graph corresponds to the activity of one major fragment.
Stair steps in the graph indicate that the execution work of the fragments is not distributed evenly. This typically occurs for non-local reads on data. To address this issue, you can increase data replication, rewrite the data, or file a JIRA to get help with the issue.
The visualized plan illustrates color-coded major fragments divided and labeled with the names of the operators used to complete each phase of the query. Exchange operators separate each major fragment. These operators represent a point where Drill can execute operations below them in parallel.
Operator Profiles
Operator profiles describe each operator that performed relational operations during query execution. The Operator Profiles section provides an Overview table of the aggregate time and memory metrics for each operator within a major fragment.
Identify the operations that consume a majority of time and memory. You can potentially modify options related to the specific operators to improve performance.
Below the Overview table are operator blocks, which you can expand to see metrics for each operator. Each of these blocks corresponds to a row in the Overview table. Each row in the Operator block presents time and memory metrics, as well as aggregate input metrics for each minor fragment.
Drill uses batches of records as a basic unit of work. The batches are pipe-lined between each operation. Record batches are no larger than 64K records. While the target size of one record batch is generally 256K, they can scale to many megabytes depending on the query plan and the width of the records.
The Max Records number for each minor fragment should be almost equivalent. If one, or a very small number of minor fragments, perform the majority of the work, there may be data skew. To address data skew, you may need change settings related to table joins or partition data to balance the work.
Data Skew Example
The following query was run against TPC-DS data:
0: jdbc:drill:zk=local> select ss_customer_sk, count(*) as cnt from store_sales where ss_customer_sk is null or ss_customer_sk in (1, 2, 3, 4, 5) group by ss_customer_sk;
|-----------------|---------|
| ss_customer_sk | cnt |
|-----------------|---------|
| null | 129752 |
| 5 | 47 |
| 1 | 9 |
| 2 | 43 |
| 4 | 10 |
| 3 | 11 |
|-----------------|---------|
6 rows selected
In the result set, notice that the ‘null’ group has 129752 values while others have roughly similar values.
Looking at the operator profile for the hash aggregate in major fragment 00, you can see that out of 8 minor fragments, only minor fragment 1 is processing a substantially larger number of records when compared to the other minor fragments.
In this example, there is inherent skew present in the data. Other types of skew may not strictly be data dependent, but can be introduced by a sub-optimal hash function or other issues in the product. In either case, examining the query profile helps understand why a query is slow. In the first scenario, it may be possible to run separate queries for the skewed and non-skewed values. In the second scenario, it is better to seek technical support.
Physical Plan View
The Physical Plan view provides statistics about the actual cost of the query operations in terms of memory, I/O, and CPU processing. You can use this profile to identify which operations consumed the majority of the resources during a query, modify the physical plan to address the cost-intensive operations, and submit the updated plan back to Drill. See Costing Information.
Canceling a Query
You may want to cancel a query if it hangs or causes performance bottlenecks. You can cancel a query from the Profile tab of the Drill Web UI.
To cancel a query, complete the following steps:
- Navigate to the Drill Web UI at
<drill_node_ip_address>:8047
. The Drill node from which you access the Drill Web UI must have an active Drillbit running. - Select Profiles in the toolbar. A list of running and completed queries appears.
- Click the query for which you want to see the profile.
- Select Edit Query.
- Click Cancel query to cancel the query.
The following message appears:
Cancelled query <QueryID\>


{{/* Copyright 2019 Google LLC Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */}}
| Emails | Actions |
|---|---|
| {{$key}} |


🗞 Newsletter
✍ Wiki
th3.
build
/tv
we do blockchain stuff & make videos
Subscribe
Make a Wallet
Bitcoin ATM
Lightning
🚫ADS
BUGS🐛
made by
phil
&
dean


Implement Singlton
3 ways of writing Singleton.
using Enum
This is only available since Java 6.
public enum Singleton_Enum {
INSTANCE;
}
using double checked locking
This is lazy loaded thread-safe Singleton, which is popular during Java 5 (with the use of Volatile variable).
public class Singleton_DoubleCheckedLocking implements Cloneable {
private static volatile Singleton_DoubleCheckedLocking INSTANCE;
private Singleton_DoubleCheckedLocking() {
}
public static Singleton_DoubleCheckedLocking getInstance() {
if (INSTANCE == null) {
synchronized (Singleton_DoubleCheckedLocking.class) {
// double checking Singleton instance
if (INSTANCE == null) {
INSTANCE = new Singleton_DoubleCheckedLocking();
}
}
}
return INSTANCE;
}
}
using static factory method
Singleton instance is static and final variable it initialized when class is first loaded into memeory so creation of instance is inherently thread-safe.
public class Singleton_StaticFactory {
// initailzed during class loading
private static final Singleton_StaticFactory INSTANCE = new Singleton_StaticFactory();
// to prevent creating another instance of Singleton
private Singleton_StaticFactory() {
}
public static Singleton_StaticFactory getSingleton() {
return INSTANCE;
}
}
About thread-safety
Prior to Java 5 double checked locking mechanism is used to create thread-safe singleton in Java, which breaks if one Thread doesn’t see instance created by other thread at same time and eventually you will end up with more than one instance of Singleton class.
From Java 5 onwards volatile variable guarantee can be used to write thread safe singleton by using double checked locking pattern.
I personally don’t prefer that way as there are many other simpler alternatives like:
- using static field
- using Enum
Q & A
Question: How do you prevent for creating another instance of Singleton using clone() method?
Answer: Preferred way is not to implement Clonnable interface. And if you do, just throw Exception from clone() method as “Can not create clone of Singleton class”.


Location: Seattle, WA
I visited and ran around Greenlake for the first time today. We did the beautiful 2.8 mile loop.
Location: Bay Area, CA
I will be heading down to the Bay Area to celebrate the birthdays of two of my friends. We will be going on a San Francisco scavenger hunt, and then capping the night off at a bar with old school games.
I just landed a job teaching high school math and science at an independent school in Seattle. I have recently been fascinated with all things concerning web development in my quest to create a personal website/blog. In addition, I have also come to enjoy traveling to other countries, visiting National Parks, and experiencing new things. I want to use this website to document all of my "firsts".


We extend networks!
- mesh plug-and-play paradigms
- visualize holistic relationships
- aggregate bricks-and-clicks systems
- revolutionize revolutionary architectures
- embrace transparent solutions
- scale dot-com infrastructures
- e-enable clicks-and-mortar supply-chains
- generate efficient portals
- deliver mission-critical content
- strategize robust content
- target cross-media niches


Beer name
Name should be between 2 and 62 characters long.
ABV
Unlike hard liquor, beers aren't measured by proof, but most commonly by alcohol by volume.
Style
1
Please select one of the listed styles or contact our administrator if you don't find it.
Brewery
1
Brewing companies range widely in the volume and variety of beer produced, ranging from small breweries, such as Ringwood Brewery, to massive multinational conglomerates.
Country
1
Where was it made?
Please describe the beer:
Create


THtmlViewer, TFrameViewer, and TFrameBrowser components
Declaration
property Palette: HPalette;
In order to best display many images each of which may have its own palette, the THtmlViewer component normally color dithers each image to a compromise rainbow palette. The Palette property allows the user to access this palette or to change it.
A common reason for wishing to change the palette is when the user has complete control of all the images to be displayed and the images all use the same palette.
Usage Notes
The Palette property has no effect in modes other than 256 color mode.
Palettes are copied when read or written.
Assume that all images will use a common palette. The following shows code in the form's OnCreate event handler which uses the palette from one of those images to change THtmlViewer's palette.
procedure TForm1.FormCreate(Sender: TObject);
var
Bitmap: TBitmap;
begin
Bitmap := TBitmap.Create;
{Load a bitmap that has the common palette}
Bitmap.LoadFromFile(ExtractFilePath(ParamStr(0))+'myimage.bmp');
{Replace the palette with the common palette}
Viewer.Palette := Bitmap.Palette;
Bitmap.Free;
{no need to dither the images now}
Viewer.Dither := False;
end;
See Also


Welcome, to the image search abstraction page
To search, simply type your search term after the / in the address bar
You can also add a ?offset=NUMBER to paginate your results to NUMBER
For example, https://cestzen-is.herokuapp.com/apple?offset=10You can also view the last ten searches by clicking here


The
section contains step-by-step tutorials, which demonstrate how to
extend the functionality of CLR applications with scripting. It will
guide you through the instantiation of the script engine (CSScriptLibrary.dll) and the execution of scripts at runtime. All tutorials are examples of script hosting according the "Type sharing" pattern.
In This Section
Text Processor
Simple "text editor" application uses the C# code entered by the user
for text manipulations. The "type sharing" pattern is implemented with
passing well-known type between the script host and the actual script.
Image Processor
Simple "image editor" application dynamically loads files containing
image manipulation C# routines and applies them to the image. The "type
sharing" pattern is implemented with passing custom type between the
script host and the actual script.


Trait SortableEntries
Direct Known Users
App\Http\Requests\PaginatedRequestIndirect Known Users
App\Http\Requests\ProjectListRequest
public
Illuminate\Database\Eloquent\Collection|static[]
|
#
getEntriesSortable( )
Helper function to return the entries, if the request has the sorted key it will sort the collection first. |
public
Illuminate\Database\Eloquent\Collection|static[]
|
|
public
Closure
|
#
getSortingFunction( )
Returns the sorting function for the request. Default is string comparison with all characters in lower case. |
public
string
|
|
public
string
|
#
getSortKeyFriendly( )
Returns the friendly name of the sort key, converting from lower underscore to upper normal case. |


I came into a cache of oxycontin 20's a while back, but I've been just holding them for awhile, waiting for a good time to try one.
I have limited experience with painkillers, mostly just taking them per doctor's prescription (no real abuse). Several times I have thrown up from them (for instance, I once threw up while sitting in the dentist's chair, and once at home after taking two hydrocodones for tooth pain). This either means I have a very low tolerance for them (very possible considering all the other substances I've puked in the past) or that I'm allergic to painkillers (which I'm not sure about).
Anyway, I gave one of the oxycontin 20's I had to a pharmacutecal-experienced friend for him to try. He crushed it and swallowed it and reported having a nice, warm and fuzzy time. So earlier this week, my boyfriend and I decided to try one, too. Around 12:30 a.m. we each swallowed a 20, uncrushed. I was a bit anxious that I might become sick from it, and wished there was some way to take a half a dose. It wasn't really possible to do this, as I wanted to avoid cutting or crushing it at all, at least for my first time.
It wasn't long before we both noticed the effects of the pill comming on. As we were both lying in bed, we could really stretch out and just relax. We didn't speak much to each other, but just kind of lazily rubbed each other's arms and shoulders. I begain feeling quite warm. After kicking off the sheets, taking off my shirt, turning up the fan, and opening the window I managed to find some short lived relief. But I found that the effects seemed to come in waves, and I was soon hot again. Other than this, the feeling was nice. Boyfriend said he felt like he was sinking into the bed. When I closed my eyes, I'd feel a bit spinny and dizzy for a few seconds, but it wasn't really bad.
We drifted into sleep eventually. Actually I think I sort of made myself go to sleep, as the effects started to seem too much for me (sleeping is my main coping mechanism in life).
Around 5:00 in the morning I woke up, still feeling hot, and a bit irritable. I roused my boyfriend and asked him to get me some water. I didn't feel so well, but I couldn't really move, so when he put the bottle to my hand, I took the cap off and just lied there, holding the bottle and not drinking. Eventually the bottle tiped and the cool water that splashed onto my skin jarred me to attention. 'oh yeah...water...', I thought. So I took a swig, lied back down, and almost immediately felt nauseous. I threw up (damn it!) in two short bursts, felt better, and lied back down. But here's the bizarre part: as I was leaned over the trash can, I noticed a pill inside. It was white, round, and completely intact. 'Did I just puke this pill?' I wondered. I couldn't see it well enough to examine it at that time, nor did I feel like it, but I gingerly picked it up and set it aside to look at later.
I went to sleep and didn't wake until 9:00 pm that night ( 18 hours - I told you I sleep a lot!). My boyfriend had no problems and woke early to go to work. He said he felt really good all day.
When I finally looked at the pill I had set aside, I could still see the OC engraved on one side and the 20 on the other. It was still very whole and un-digested looking, which surprised me. As much as I felt from the pill, I would have expected it to have broken down, and really gotten in my system. The only difference is that it was white, whereas the unswallowed ones are pink (a coating, I suppose). Maybe it didn't break down due to its time-release properties; I'm not sure.
I felt good until I got sick. I didn't even feel that terribly sick, compared to other times I've been sick. The next day I felt dizzy, groggy, and generally out of it. All in all, it was a mixed experience.
Experience Reports are the writings and opinions of the individual authors who submit them.
Some of the activities described are dangerous and/or illegal and none are recommended by Erowid.


Client Errors¶
The description of client side errors that can get from REST API server on known logic errors and unknown exceptions.
Error Model¶
Represents the Error model as value object with available properties.
| Name | Description | Type |
|---|---|---|
id |
Representing Error identifier from API Logs. | guid |
message |
The Error message it can be descriptive for known error and generic for unknown exceptions. | string |
description |
The Error description mainly a generic text for unknown exception. | string |
httpStatusCode |
The HTTP Status Code, eg. 200. | integer |
requestArguments |
The collection of sent Request Arguments. | array(RequestArgument) |
Hint
The Error properties names (Name column) is for default usage by JSON
, for C# Wrapper
usage the Error properties are capitalized (eg. Id
, Name
,..)!
Note that C# Wrapper has for Error Model overridden a ToString()
method and calling instance of error like error.ToString()
will result with displaying the Error and all Requested Arguments
Warning
Note that only simple Request Arguments will be sent like integer
, string
, boolean
,... The complex types are currently not supported to be sent as Error Request Arguments.
Request Argument Model¶
Represents the Request Argument model as value object with available properties.
| Name | Description | Type |
|---|---|---|
name |
The Request Argument name, eg. id . |
string |
value |
The Request Argument value for name Request Argument. |
string |
Hint
The Error properties names (Name column) is for default usage by JSON
, for C# Wrapper
usage the Error properties are capitalized (eg. Id
, Name
,..)!
Known Errors¶
The list of Known Errors that can happen while interacting with Coach REST API.
Hint
Note that this errors are generic description of server side known errors and errors sent to client will be with proper and very descriptive error messages, due to some logic and according to what arguments are sent when error occurred. Many errors are more described in particular entity’s documentation, mostly around its Domain Model.
| Known Error | Description |
|---|---|
| Missing Consumer Key and/or Secret | Consumer key and/or secret is not sent with HTTP Header. |
| Unauthorized Key and/or Secret | Consumer key/secret combination is not known and hence unauthorized. |
| Invalid Tenant Code | The tenant code provided through URL is not valid. Must be 1000 and greater! |
| Unknown Tenant Code | The tenant code is not existing. |
| Unknown Content-Type | The Request Header must include a valid and supported Content-Type media format (JSON or XML). |
| Exceeded License Total For Agents | The licenses exceeded the total of purchased licenses. Note one active user as agent is one license! |
| Unknown Reference Entity Id | The sent referenced entity GUID Id is unknown or empty GUID. Reference to entity can be established. |
| Unique Username | Validates the uniqueness of User’s username. There cannot be two same usernames for tenant. |
| Required Field | Validates that required field or property has proper value assigned to it. |
| Invalid Id | The Id of entity is empty GUID. |
| Invalid Date Range | The date range of start and end date is invalid when end date is greater than start date. |
| Incorrect Period Type | The incorrect combination usage of schedule period type and occurrence. |
| Incorrect Numeric | The applied numeric value is not correct whether of crossing some range, negative number, greater than 0,... |
| Incorrect Date | The applied data is no correct due some date logic. |
| Delete While Having Reference | The entities that are referenced by some other entities are not allowed to be deleted. |


The map above will let you see helpful overlays based on recorded location data. Route view is a visual trace of a previous route taken. In this view, each point along the path provides vehicle metrics recorded in that instance as a sort of snapshot of your drive. Heatmap view highlights the most travelled roads across all aggregate routes.
Two different info panels can be selected above to show relevant data recorded by your vehicle. Route Metrics displays a 2D graph of a given route with side-by-side comparisons of metrics like speed and RPMs. The Calendar is both a historical log of previous drives and a handy schedule to track upcoming reminders of any necessary vehicle maintenance.
Transportation research is an ongoing challenge of trying to understand traffic data and apply it in a way that makes our public systems more efficient. Every driver on the road today has thought about how to get to their destination more efficiently; many researchers try to find ways to mitigate traffic for reasons such as emissions and efficiency. We want to provide a tool to help quantify those efforts and model them.
TraceRT ("Trace Route") is a vehicle blackbox system composed of an Android app, a cloud database, and this web interface. The app takes data from an OpenXC OBD 2 sensor, the Google Maps API, and the sensors built into your mobile device, and sends it to our Firebase database. This provides users with data to model things like vehicle statistics, emissions, fuel efficiency, route comparisons, and even keeping track of car service records for regular maintenance. Users can also download their data directly for other applications.
The mobile Android app component of our project was developed in Android Studio. Java is the supporting languages for the backend of the application, while XML (eXtensible Markup Language) is used to define the user interface. The app is seen as the bridge between the OBD 2 Sensor, our cloud database, and phone sensors. With that being said, the app has many processes going on at once, so we've optimized the code to prevent runtime crashes and memory loss.
The web interface serves as a frontend of our system by giving users a clear view of data models and conceals the underlying data retrieval, processing, and hardware interaction handled by other components of the blackbox system. Web hosting is done through GitHub Pages and dynamic data visualization is handled by the D3 Javascript library. Users are able to login through their Google accounts to see information from their own vehicles. Data collected by the app is pulled from our database to create informative visualizations of each data model in our system: route efficiency, emissions, road quality, and maintenance logging and tracking. This data can be overlaid upon maps to add another dimension of location information.
This project was created for our Computer Science senior design class at Texas A&M University. In our team of 3, each member worked on different components of the system, dividing up our responsibilities. Caleb E., our team leader, configured the OpenXC hardware and set up our database, as well as team management tasks. Ricky S. designed and programmed our Android app and handled interconnectivity between the hardware, device, and database. Jared M. designed and implemented our website and data visualizations.
Visit our project on GitHub!
Web Interface
Android App *
* Note: requires TAMU login


Currency Conversion Made Easy
Instructions: Input USD on the left, hit the convert button below to return current exchange rate.


BOSH is a Cloud Foundry project for release engineering, deployment, and lifecycle management of large-scale cloud software. BOSH can provision and deploy software over hundreds of VMs and performs monitoring, failure recovery and software updates with zero-to-minimal downtime.
BOSH was developed to deploy Cloud Foundry, but can also be used to deploy other software like Hadoop and is well-suited for large distributed systems. BOSH supports Infrastructure as a Service (IaaS) providers VMware vSphere, vCloud Director, Amazon Web Services EC2, and OpenStack. There is a Cloud Provider Interface (CPI) that has enabled users to extend BOSH to support additional IaaS providers such as Google Compute Engine and Apache CloudStack.
BOSH has an active community on Google Groups. If you have questions, try either the BOSH Users mailing list or if you are developing or extending BOSH try the BOSH Developers mailing list. If you're interested in contributing to the project, read the contribution guidelines. If you have security concerns, please contact the core team directly at security@cloudfoundry.org.


TypeScript is a typed superset of JavaScript that compiles to plain JavaScript.
Any browser. Any host. Any OS. Open source.
For NPM users:
> npm install -g typescript
Provides a command-line compiler
> tsc source.ts
function add(a: number, b: number) {
return a + b;
}
let decimal: number = 6;
let isDone: boolean = false;
let color: string = "blue";
let u: undefined = undefined;
let n: null = null;
const reports: Employee[] = [];
All types are subtypes of a single top type called the Any type.
let list: Array<number> = [1, 2, 3];
let list: number[] = [1, 2, 3];
let notSure: any = 4;
notSure = "maybe a string instead";
notSure = false;
let unusable: void = undefined;
function add(x: number, y: number): number {
return x + y;
}
let myAdd = function(x: number, y: number): number {
return x + y;
};
let myAdd: (baseValue: number, increment: number) => number =
function(x: number, y: number): number { return x + y; };
function buildName(firstName: string, lastName?: string) {
if (lastName)
return firstName + " " + lastName;
else
return firstName;
}
let result1 = buildName("Bob");
let result2 = buildName("Bob", "Adams", "Sr.");
let result3 = buildName("Bob", "Adams");
function buildName(firstName: string, lastName = "Smith") {
return firstName + " " + lastName;
}
let result1 = buildName("Bob");
let result2 = buildName("Bob", undefined);
let result3 = buildName("Bob", "Adams", "Sr.");
let result4 = buildName("Bob", "Adams");
const reports: Employee[] = [];
All types are subtypes of a single top type called the Any type.
function getId(id: T): T {
return id;
}
let result1 = getId<number>(5);
console.log(result1);
let result2 = getId<string>("abc");
console.log(result2);
class User {
private _name: string;
private _age: number;
constructor(name: string, age: number) {
this._name = name;
this._age = age;
}
public displayInfo(): void {
console.log("name: " + this._name + ";
age: " + this._age);
}
}
interface IUser {
id: number;
name: string;
}
let employee: IUser = {
id: 1,
name: "Alice"
}
function getEmployeeInfo(user: IUser): void {
console.log("id: " + user.id);
console.log("name: " + user.name)
}
getEmployeeInfo(employee);
export interface Device{
name: string;
}
export class Phone implements Device {
name: string;
constructor(n:string){
this.name = n;
}
}
export function Call(phone: Phone) : void{
console.log("Make a call by", phone.name);
}
import Entry from "./components/Entry";
import { DefaultHeaders } from "../constants/http";
import "/mod";
import * as $ from "jquery";
import { Component } from "@angular/core";


Percentage distribution of women aged 18 years and older who were married or in union before age 18 by region from a survey taken in 2013
Percentage of women aged 20 to 49 years who were married or in union before ages 15 and 18, in the 10 countries with the highest prevalence of child marriage
The dark red = "Girls married before the age of 15"
The light red = "Girls married after the age of 15 but before the age of 18"
In February 2016, the Girls Not Brides team headed out to Zambia to film “We are Girls, Not Brides”, a two-minute music video about ending child marriage. The song was written, composed and sung by a group of talented young girls from Lusaka Girls School, who take part in a girls’ empowerment club run by Girls Not Brides member Continuity Zambia.
Photographer Stephanie Sinclair and writer Cynthia Gorney investigate the world of prearranged child marriage, where girls as young as five are forced to wed.


--- title: "Lab 8: Instrumental variables" linktitle: "Lab 8" date: "2020-10-14" due_date: "2020-11-23" due_time: "2:20 PM" menu: assignment: parent: Labs weight: 7 type: docs output: blogdown::html_page: toc: true editor_options: chunk_output_type: console ---
It’s our final lab of the semester! :raised_hands: :trophy: :dancer: :clap:
Do-file template labtemplate.do
By the end of this lab, you should be able to complete the following tasks in Stata:
Estimate instrumental variable specifications and interpret them.
Output regression results using outreg2
ivregress
We can estimate an instrumental variables regression with ivregress
General form:
ivregress estimator depvar [varlist1] (varlist2 = varlist_iv) [if] [in] [weight] [, options]
estimator
is where we will type 2sls
depvar
is your dependent variableExample:
To estimate the following two-stage least squares equation: \[ rent = \beta_0 + \beta_1 \widehat{hsngval} + \beta_2 pcturban + u\] where \(\widehat{hsngval}\) is predicted from the following first-stage equation \[ hsngval = \alpha_0 + \alpha_1 faminc + \alpha_2 pcturban + v \]
webuse hsng2
ivregress 2sls rent (hsngval = faminc ) pcturban, robust
You can add , first
to report the first-stage results:
`ivregress 2sls rent (hsngval = faminc ) pcturban, robust first`
outreg2
We are very good at reading raw Stata output. But, raw stata output has no place in our papers. How do we make it pretty? There are lots of ways, including putexcel
, which lets you create customizable excel tables with your outputs (good for descriptive statistics), and estout
, which does the same thing but is more regression oriented.
Personally, I like outreg2
, because it’s easy to set up and use.
You’ll run outreg2
after estimating a regression. It takes your results and saves them to a table. You can run it multiple time and generate columns of results within the same excel sheet, which is pretty handy! The general format of outreg2 is this:
// You can copy and paste this into stata, and it should work! But note that it will save to your working directory
sysuse auto,clear
// Specification 1
regress mpg foreign weight headroom trunk length turn displacement
outreg2 using myfile.xls, replace
// Specification 2 (add on)
regress mpg foreign weight headroom trunk length turn displacement,robust
outreg2 using myfile.xls, append
You can customize, with lots of options! (see help outreg2
, or check out these resources)
What sort of things?
outreg2 using myfile, word replace
outreg2 using myfile, addnote(Dummy variables not shown)
outreg2 using myfile, keep(mpg foreign)
outreg2 using myfile, dec(5)
An example:
sysuse auto,clear
local r "replace"
forval num=1/5 {
regress mpg weight headroom if rep78==`num'
sum mpg if rep78 == `num'
local mean = `r(mean)'
outreg2 using myfile.xls, `r' keep(headroom) title("Sample Graph") nocons addtext("Rep78", `num') addstat("Mean", `mean') auto(2) bracket
local r "append"
}
Today we’re going to work with voucher.dta
, a data set of student
performance from Rouse (1998). She measures the impact of private school
vouchers on student achievement. The final measure of student
performance we’re interested in is mnce
, their math test scores in
1994 (after up to four years in the private school). We also have some
measures of baseline performance, their math test score in 1990
(mnce90
). The variable choiceyrs
is the number of years enrolled in
a private school, and selectyrs
is the number of years a student was
selected to receive a voucher to fund enrolling in a private school.
In your do-file, start a log and open voucher.dta
.
Summarize your data. Of the 990 students in the sample, how many were never awarded a voucher? How many had a voucher for all four years? How many actually attended a choice school for four years?
Predict the relationship between choice school attendance and math
scores by regressing math scores mnce
(dependent variable) on
number of years enrolled in a choice school choiceyrs
(independent
variable). What do you find? Is this what you expect? What happens
if you add in the variables black
, hispanic
, and female
? Write
your results in equation form.
Why might choiceyrs
be endogenous? Explain:
Now, estimate a regression of \(choiceyrs\) (dependent variable) on
\(selectyrs\) (independent variable), including race/ethnicity and
gender controls. Why is this a reasonable choice of an instrument?
What is the F-statistic on selectyrs
? (Hint: You can use the
testparm
command for a hypothesis test with just one coefficient)
Based on the previous regression, use the predict
command to
generate a predicted \(\widehat{choiceyrs}\). Estimate the regression
of \(mnce\) on \(\widehat{choiceyrs}\), including race/ethnicity and
gender controls. Write the estimated equation. How does your result
compare to your OLS estimate?)
Re-estimate a regression of \(mnce\) (dependent variable) on
\(choiceyrs\) (independent variable) using \(selectyrs\) as an
instrument for \(choiceyrs\). However, this time, estimate the
equation in one command line using ivregress 2sls
. How do your
results change, if at all?
Repeat your IV analysis, but this time include a control for baseline achievement by adding \(mnce90\). Write the results in equation form below. Do you find these results convincing? Explain.
We can also use multiple instruments for multiple endogenous variables. The variables \(choiceyrs1\), \(choiceyrs2\), etc. are dummy variables indicating the different number of years a student could have been in a choice school. Similarly, \(selectyrs1\), \(selectyrs2\), etc. have a similar definition, but for being selected from the lottery.
Estimate the following equation using IV. \[\begin{split} mnce &= \beta_0 + \beta_1 choiceyrs_1 + \beta_2 choiceyrs_2 + \beta_3 choiceyrs_3 + \beta_4 choiceyrs_4 + \\ & \beta_5 black + \beta_6 hispanic + \beta_7 female + \beta_8 mnce90 + u \end{split}\]
Finally, go back through your regressions in your do-file. After
each regression (there should be six: OLS without controls, OLS with
controls, IV by hand, IV using ivregress
, IV with \(mnce90\), and IV
with multiple instruments), add a line of code to output the results
to a word or excel file using outreg2
.
Include a table with your results with your submission - there
should be six columns in one table. Note that you can use the
append
option to add each regression as a new column, rather than
a new file.
References: Rouse, Cecilia Elena (1998), “Private School Vouchers and Student Achievement: An Evaluation of the Milwaukee Parental Choice Program,” The Quarterly Journal of Economics 113(2), 553-602.


{% extends '_main.html' %} {% block breadcrumbs %}
You don't have any subscriptions right now.
You can unsubscribe or change your email preferences at any time.
| Name | Changes | ||
|---|---|---|---|
| {{ subscription.dataset.name }} |
{% for option in subscription.get_list_of_selected_options %}
{{ option }} {% endfor %} |
Change preferences | Unsubscribe |


Beta limit means there is a limit for the value of beta beyond which the plasma will encounter a serious disruption. Early calculation of the beta limit on JET shows that the maximal βt obtained is proportional to Ip∕(106aBt0), where all quantities are in SI units. This scaling relation βt max = CTIp∕(106aBt0) is often called Troyon scaling, where the coeﬃcient CT was determined numerically by Troyon to 0.028. Often CT is expressed in percent, in which case CT = 2.8. This motivates us to deﬁne
|
| (537) |
which is called “normalized beta”. The normalized beta βN is an operational parameter indicating how close the plasma is to reach destabilizing major MHD activities. Its typical value is of order unit.
As mentioned above, βN calculated by Troyon is 2.8. Empirical evaluation from the data of diﬀerent tokamaks raises this value slightly to 3.5, although signiﬁcantly higher values, e.g., βN = 7.2, have been achieved in the low aspect ratio tokamak NSTX[22].
The value of βN indicates how close one is to the onset of deleterious instability . The ability to increase the value of βN can be considered to be the ability of controlling the major MHD instabilities, and thus can be used to characterize how well a tokamak device is operated. One goal of EAST tokamak during 2015-2016 is to sustain a plasma with βN ≥ 2 for at least 10 seconds.
(check** The tearing mode, speciﬁcally the neoclassical tearing mode (NTM) is expected to set the beta limit in a reactor.)
(**check: Tokamak experiments have found that it is easier to achieve high βN in large Ip plasmas than in small Ip plasmas. However, experiments found it is easier to achieve high βp in small Ip plasmas than in large Ip plasmas. Examining the expression of βN and βp given by Eqs. (536) and Eq. (??), respectively, we recognize that pressure limit should have a scaling of ⟨p⟩∝ Ipα with 1 < α < 2. )


net.jini.core.lease
Interfaces
Lease
LeaseMap
Exceptions
LeaseDeniedException
LeaseException
LeaseMapException
UnknownLeaseException


Presented by Blake Draper - USGS Web Informatics and Mapping (WiM)
a system for collection, management and dissemination of flood data
FEV is the public data viewing tool for the STN database
diagram of STN universe
USGS Office of Surface Water
POC: Marie Peppler / Todd Koenig
Funded largely by Hurricane Sandy money
Hurricane Irene Mapper (2011)
Version 1, born ~2012
The Release
USGS Storm Team activated October 3rd
FEV v2 alpha released later that day
FEV 2.0 offically released October 5th
Matthew arrives October 7th
used extensively by USGS personnel
designed with public in mind
FEV - indefinitely
this version - 4-5 years?
pretty basic because just JS/jQuery/Bootstrap
src JS split into 3 files
gulp build system
Questions?


API Namespaces¶
INK APIs are grouped into namespaces according to their different functionalities. In some cases, namespaces can be further divided into categories. Check out the available INK namespaces and their corresponding APIs below:
EDITOR INKAPI.editor
¶
- getAllBlocks
EDITOR
- getAuthStatus
EDITOR
- getHTML
EDITOR
- getMD
EDITOR
- getSelectedData
EDITOR
- getText
EDITOR
- insertBlock
EDITOR
- insertImage
EDITOR
- insertText
EDITOR
- insertTextAtCursor
EDITOR
- loadHTML
EDITOR
- loadText
EDITOR
- off
EDITOR
- on
EDITOR
- removeBlock
EDITOR
- setBlockData
EDITOR
- setSelection
EDITOR


Classes
The following classes are available globally.
-
A Bytes object represents an array of UInt8 and provides various utilities for importing and exporting values into and out of that array. A Bytes object maintains a position marker which is used to denote the position from which new export operations proceed. An export will advance the position by the appropriate amount.
See moreDeclaration
Swift
public final class Bytes
-
Provides access to a file on the local file system
See moreDeclaration
Swift
public class File
-
Undocumented
See more
-
Stand-alone HTTP server. Provides the same WebConnection based interface as the FastCGI server.
See moreDeclaration
Swift
public class HTTPServer
-
This non-instantiable object serves as an access point to a registry for JSONConvertibleObjects A JSONConvertibleObject is a custom class or struct which can be converted to and from JSON.
See moreDeclaration
Swift
public class JSONDecoding
-
Base for a custom object which can be converted to and from JSON.
See moreDeclaration
Swift
public class JSONConvertibleObject: JSONConvertible
-
This class is responsible for reading multi-part POST form data, including handling file uploads. Data can be given for parsing in little bits at a time by calling the
See moreaddTobuffer
function. Any file uploads which are encountered will be written to the temporary directory indicated when theMimeReader
is created. Temporary files will be deleted when this object is deinitialized.Declaration
Swift
public final class MimeReader
-
This class represents an individual scope for mustache template values. A mustache template handler will return a
See moreMustacheEvaluationContext.MapType
object as a result from itsPageHandler.valuesForResponse
function.Declaration
Swift
public class MustacheEvaluationContext
-
An instance of this class will collect all output data generated by mustache tags during evaluation. Call the
See moreasString()
function to retreive the resulting data.Declaration
Swift
public class MustacheEvaluationOutputCollector
-
An individual mustache tag or plain-text section
See moreDeclaration
Swift
public class MustacheTag
-
A sub-class of MustacheTag which represents a mustache
See morepartial
tag.Declaration
Swift
public class MustachePartialTag : MustacheTag
-
A sub-class of MustacheTag which represents a pragma tag. Pragma tags are
See moremeta
tags which influence template evaluation but likely do not output any data.Declaration
Swift
public class MustachePragmaTag : MustacheTag
-
A sub-class of MustacheTag which represents a group of child tags.
See moreDeclaration
Swift
public class MustacheGroupTag : MustacheTag
-
This class represents a mustache template which has been parsed and is ready to evaluate. It contains a series of
See moreout of band
pragmas which can influence the evaluation, and a series of children which constitute the body of the template itself.Declaration
Swift
public class MustacheTemplate : MustacheGroupTag
-
This object will parse templates written in the mustache markup language. Calling
See moreparse
with the path to a file will return the resulting parsed and ready to evaluate template.Declaration
Swift
public class MustacheParser
-
This class permits an external process to be launched given a set of command line arguments and environment variables. The standard in, out and err file streams are made available. The process can be terminated or permitted to be run to completion.
See moreDeclaration
Swift
public class SysProcess


Welcome! Let's start cooking!
Today we're cooking moist muffins, a very delicious desert that can be not only enjoyed alone but also combined with a glass of milk or orange juice.
Ingredients:
- 2 eggs
- 110g. melted butter
- 110g. sugar
- 140g. flour
- 1 pouch of Vanilla Sugar
- 2 tsp baking powder
- a few drops of rum essence
- chopped chocolate
Instructions:
- Mix the dry ingredients: Whisk the flour and baking powder in a large bowl. Set aside.
- Mix the wet ingredients: Whisk the eggs, sugar and butter until combined.
- Fold the dry ingredients into the wet: Carefully add the dry ingredients to the wet ones and mix with a wooden spoon or rubber spatula until just combined.
- Fold in the mix-ins: All that's left to do is adding the chocolate and the essence.
- Prepare the pan and oven: Arrange a rack in the middle of the oven and heat to 220oC. Line a standard 12-well muffin pan with papers liners
- Fill the pan and bake: Divide the batter amongst the prepared muffin wells, about 1/3 cup of batter per well. Bake at 180oC until the muffins are golden brown and a toothpick inserted into the center of a muffin comes out clean, about 30 minutes.


My Projects
Coding Dojo
Voxel Model Editor
3D Raytracer
Cellular Automata
Langton Ant
Procedurally Generated Maps


Esprima ([esprima.org](http://esprima.org), BSD license) is a high performance, standard-compliant [ECMAScript](http://www.ecma-international.org/publications/standards/Ecma-262.htm) parser written in ECMAScript (also popularly known as [JavaScript](http://en.wikipedia.org/wiki/JavaScript>JavaScript)). Esprima is created and maintained by [Ariya Hidayat](http://twitter.com/ariyahidayat), with the help of [many contributors](https://github.com/ariya/esprima/contributors).
### Features
- Full support for ECMAScript 5.1 ([ECMA-262](http://www.ecma-international.org/publications/standards/Ecma-262.htm))
- Sensible [syntax tree format](http://esprima.org/doc/index.html#ast) compatible with Mozilla
[Parser AST](https://developer.mozilla.org/en/SpiderMonkey/Parser_API) - Optional tracking of syntax node location (index-based and line-column) - Heavily tested (> 650 [unit tests](http://esprima.org/test/) with [full code coverage](http://esprima.org/test/coverage.html)) - [Partial support](http://esprima.org/doc/es6.html) for ECMAScript 6
Esprima serves as a building block for some JavaScript language tools, from [code instrumentation](http://esprima.org/demo/functiontrace.html) to [editor autocompletion](http://esprima.org/demo/autocomplete.html).
Esprima runs on many popular web browsers, as well as other ECMAScript platforms such as [Rhino](http://www.mozilla.org/rhino), [Nashorn](http://openjdk.java.net/projects/nashorn/), and [Node.js](https://npmjs.org/package/esprima).
For more information, check the web site [esprima.org](http://esprima.org).


@* * Copyright 2023 HM Revenue & Customs * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. *@ @import controllers.routes.{MovementDetailsController, SpecificDateTimeController} @import forms.{Location, SpecificDateTimeChoice} @import models.requests.JourneyRequest @import uk.gov.hmrc.govukfrontend.views.html.components._ @import views.components.BackButton @import views.html.components.goodsLocationExpander @import views.html.components.gds._ @import views.Title @this( govukLayout: gds_main_template, exportsInputText: exportsInputText, goodsLocationExpander: goodsLocationExpander, errorSummary: errorSummary, sectionHeader: sectionHeader, formHelper: FormWithCSRF, saveButtons: saveButtons ) @(form: Form[Location], consignmentReference: String, specificDateTimeChoice: Option[SpecificDateTimeChoice])(implicit request: JourneyRequest[_], messages: Messages) @backUrl = @{specificDateTimeChoice.map(_.choice) match { case Some(SpecificDateTimeChoice.CurrentDateTime) => SpecificDateTimeController.displayPage case _ => MovementDetailsController.displayPage }} @heading = {


Should hyphenated arguments be expanded into camel-case aliases? Default is true
Should arrays be combined when provided by both command line arguments and a configuration file. Default is false
Should keys that contain . be treated as objects? Default is true
Should arguments be coerced into an array when duplicated. Default is true
Should array arguments be coerced into a single array when duplicated. Default is true
Should parsing stop at the first text argument? This is similar to how e.g. ssh parses its command line. Default is false
The prefix to use for negated boolean variables. Default is 'no-'
Should keys that look like numbers be treated as such? Default is true
Should unparsed flags be stored in -- or _. Default is false
Should a placeholder be added for keys not set via the corresponding CLI argument? Default is false
Should a group of short-options be treated as boolean flags? Default is true
Should aliases be removed before returning results? Default is false
Should dashed keys be removed before returning results? This option has no effect if camel-case-expansion is disabled. Default is false
Generated using TypeDoc
Should variables prefixed with --no be treated as negations? Default is
true


def max(x, axis=None, keepdims=False):
if isinstance(axis, list):
for a in axis:
x = np.max(x, axis=a, keepdims=keepdims)
return x
else:
return np.max(x, axis=axis, keepdims=keepdims)
def max(x, axis=None, keepdims=False):
if isinstance(axis, list):
axis = tuple(axis)
return np.max(x, axis=axis, keepdims=keepdims)
def min(x, axis=None, keepdims=False):


Components
You load a component through the @Component
decorator. It accepts similiar
parameter as the @LoadHTML
decorator like selector,path and data.
A component reads a file(path) or a tag(selector).
Structure of a component
- <template> or

</template>
...

@Component(selector=#componenttest)

Create a component in an external file
You create a file called componenttest.html
and put in the following:

@Component(path=componenttest.html)

Some paragraph to pass into the component

Events
In components you have the ability to create and send events. They are several methods available:
Note: The events will be reset after rendering. So you should register the events on the updated
method.
You create an event with the on
method.
this.on('someEvent', function (file,sender) {
console.log('fileSelectSelected',file,sender);
// do stuff
});
If you have nested components you can trigger events with emit
.
The event will be bubbling upwards to the initial component.
...
var file = e.currentTarget.files[0];
this.fileNameElm.innerText = file.name;
this.emit('fileSelectSelected',file);
...
If you want events from other components or global events to react you use
the broadcast
method. You use it like the emit
method.
...
var someData = {
state : 1
};
this.broadcast('someEvent',someData);
...
After creating a component you can wrap your component into a decorator.
var mynamespace = {};
class Test extends HTMLDecorators.Decorator {
// add this to the decorator
compWrapper = true;
initialized() {
this.component = this.createDecorator('Component', HTMLDecorators.StdDecorators.Component,Object.assign({
fromStack : 'appStack',
withId : 'fileSelect'
}, this.config));
}
async render() {
await this.component.render();
}
}
mynamespace.Test = Test;
...
Now you should be able to call component by using
@mynamespace.Test


--- layout: post title: "Bayesian workflow with TFP and arviz" permalink: /:title/ tags: [tensorflow, arviz] ---
This post hopefully contains an end-to-end example of a Bayesian workflow for a simple model on some simulated data using TFP and arviz. For a more comprehensive guide on such a workflow, see e.g. Towards A Principled Bayesian Workflow.
I am hoping to update this post as I find better ways of doing this and new things are added to TF/TFP/arviz. At the time of writing, this post isn't quite fleshed out yet.
Just some imports and constants, skip this.
import logging import arviz as az import matplotlib.pyplot as plt import numpy as np import tensorflow as tf import tensorflow_probability as tfp from tensorflow_probability.python.internal import unnest plt.rcParams["figure.figsize"] = (10, 5) tf.get_logger().setLevel(logging.ERROR) tfd = tfp.distributions tfl = tf.linalg N = 2_500 P = 5 NUM_CHAINS = 8 NUM_RESULTS = 1_000 NUM_BURNIN_STEPS = 1_000 NUM_PRIOR_SAMPLES = 4096 INITIAL_STEP_SIZE = 1.0 PROPORTION_ADAPTION_STEPS = 0.8
print(f"arviz version: {az.__version__}") print(f"TF version: {tf.__version__}") print(f"TFP version: {tfp.__version__}")
arviz version: 0.10.0 TF version: 2.4.0 TFP version: 0.12.1
The model is a boring linear regression:
\begin{align*} \sigma &\sim Exponential(1) \\ \alpha &\sim Normal(0, 2.5^2) \\ \beta &\sim Normal(0, 1.0^2) \\ Y &\sim Normal(\alpha + X \beta, \sigma^2) \\ \end{align*}for some design matrix containing \(P\) features/covariates/predictors/whatever denoted by \(X\), and we have \(N\) observations denoted by \(Y\). In code:
def make_joint_dist(design_matrix): def joint_dist(): intercept = yield tfd.Normal(loc=0.0, scale=2.5, name="intercept") coefficients = yield tfd.Normal(loc=tf.zeros(P), scale=1.0, name="coefficients") observation_scale = yield tfd.Exponential(rate=1.0, name="observation_scale") yield tfd.Normal( loc=intercept + tfl.matvec(design_matrix, coefficients), scale=observation_scale, name="observations", ) return tfd.JointDistributionCoroutineAutoBatched(joint_dist) design_matrix = tfd.Normal(loc=0.0, scale=1.0).sample([N, P]) joint_dist = make_joint_dist(design_matrix)
I use a function to create the joint distribution here so that it's easy to make new joint distributions for different design matrices, we can see how that works at the end once we have some posterior samples.
Taking a sample from the joint distribution object will give us some simulated observations and the parameters which generated them:
data = joint_dist.sample() print(data)
StructTuple( intercept=<tf.Tensor: shape=(), dtype=float32, numpy=-0.20901993>, coefficients=<tf.Tensor: shape=(5,), dtype=float32, numpy= array([-2.3835316 , -0.19555189, 0.3054796 , 0.030938 , 0.39104107], dtype=float32)>, observation_scale=<tf.Tensor: shape=(), dtype=float32, numpy=1.8365525>, observations=<tf.Tensor: shape=(2500,), dtype=float32, numpy= array([ 4.681982 , 0.494495 , 3.3885002, ..., 3.0582027, 1.5004369, -0.6779976], dtype=float32)> )
Working with simulated data like this is nice as you can easily sense check your inference by comparing your estimated posterior and the "true" parameter values.
In most circumstances you of course don't know the underlying mechanism which has generated the data, and you don't know the true values of the parameters which generated it - this is just an easy way to simulate some data and parameters for an example.
One of the first things to consider doing is a prior predictive check. This involves sampling observations from our generative model without conditioning on any data. It's good to see that the observations generated from the model aren't completely ridiculous and also to catch any coding errors early on. It's also really easy to do so there's no excuse really:
*prior_samples, prior_predictive = joint_dist.sample(NUM_PRIOR_SAMPLES) print(prior_predictive)
tf.Tensor( [[-3.5205534 -4.528192 0.25591993 ... -4.3896685 -1.5770044 -0.7582612 ] [-0.42647317 0.71366215 3.773345 ... -0.9740752 -1.5424446 4.6503186 ] [ 4.607047 3.3693523 6.6304727 ... 4.502372 7.3991976 6.2240896 ] ... [-3.2436218 -0.72438574 -1.5025095 ... -1.8662915 -3.8655753 0.99725056] [ 3.0118685 3.1012888 -1.0712335 ... 3.4267337 0.48165292 -1.1030183 ] [ 2.1230254 2.7200956 1.4965906 ... 3.4740658 1.3719496 1.5941529 ]], shape=(4096, 2500), dtype=float32)
To plot the check we can quickly create an arviz.InferenceData object which just
contains the prior predictive samples and the "real" observations. I always like to give
names to the dimensions where possible and think it helps with readability. One other
comment on this is that arviz expects shapes to be of the form: (chain, draw, *shape)
so we reshape the prior predictive samples to pretend they have one chain:
prior_trace = az.from_dict( observed_data={"observations": data.observations}, prior_predictive={"observations": prior_predictive[tf.newaxis, ...]}, coords={"observation": np.arange(N)}, dims={"observations": ["observation"]}, ) print(prior_trace.prior_predictive)
<xarray.Dataset> Dimensions: (chain: 1, draw: 4096, observation: 2500) Coordinates: * chain (chain) int64 0 * draw (draw) int64 0 1 2 3 4 5 6 ... 4090 4091 4092 4093 4094 4095 * observation (observation) int64 0 1 2 3 4 5 ... 2495 2496 2497 2498 2499 Data variables: observations (chain, draw, observation) float32 -3.521 -4.528 ... 1.594 Attributes: created_at: 2021-01-06T22:23:41.549424 arviz_version: 0.10.0
Once that object is created, using arviz to do a basic prior predictive check using arviz.plot_ppc is easy:
ax = az.plot_ppc(prior_trace, group="prior", num_pp_samples=500)
Thankfully this check looks OK - given how we simulated the data anything else would be either very unlucky or a bug.
Now that the prior predictive check is done, we can move onto a harder problem - conditioning on the data and estimating the posterior distribution of the model parameters. Much of the code here is boilerplate and could be lifted into a more generic function/library.
My current favourite way to condition on the "observed" data is to create a (currently experimental) JointDistributionPinned, taken from the documentation:
This object represents an unnormalized probability density, and as such is not a
tfp.distributions.Distribution
, and lackssample
andlog_prob
methods. In their place, it provides:
unnormalized_log_prob
,unnormalized_log_prob_parts
sample_unpinned
,sample_weighted
pinned_joint_dist = joint_dist.experimental_pin(observations=data.observations) print(pinned_joint_dist.sample_unpinned())
StructTuple( intercept=<tf.Tensor: shape=(), dtype=float32, numpy=1.5575541>, coefficients=<tf.Tensor: shape=(5,), dtype=float32, numpy= array([-1.2378179 , 0.31099358, 1.6653953 , 0.20364931, 0.66097695], dtype=float32)>, observation_scale=<tf.Tensor: shape=(), dtype=float32, numpy=0.18035434> )
The first use of this pinned distribution object is to provide some default bijectors
which allow our MCMC routine to operate on the unconstrained space even though e.g. our
observation scale (\(\sigma\)) is positive. Stuff like this is handled automatically in
Stan when you specify bounds on parameters with e.g real<lower = 0> sigma;
. These
default bijectors can be extracted for all the distributions with simply:
pinned_joint_bijector = pinned_joint_dist.experimental_default_event_space_bijector()
The second use is to provide a function which returns the unormalised log probability of the model parameters (conditioned on the data):
def target_log_prob_fn(*x): return pinned_joint_dist.unnormalized_log_prob(x)
In this example we use the No-U-Turn Sampler which can record/trace various useful bits of auxiliary information - useful for checking e.g. converge of the MCMC routine. TFP allows us to add a so-called trace function which enables us to record anything we want about the current state of the chain or the NUTS routine. At the moment I'm only recording three statistics and naming them as per the arviz sample stats.
def trace_fn(state, kernel_results): mapping = { "target_log_prob": "lp", "has_divergence": "diverging", "energy": "energy", } return {v: unnest.get_innermost(kernel_results, k) for k, v in mapping.items()}
There are plots later which use some of these values.
The unnest
module isn't part of the public TFP API - but it does make writing this
function a lot easier.
When doing our prior predictive check we actually sampled many times from the prior, so one way of initialising the MCMC routine is to use some of those (one for each chain):
initial_state = [sample[:NUM_CHAINS] for sample in prior_samples]
In some cases this might not be a good idea, in particular if you have vague priors.
We also need to choose an initial step size, one of the parameters of Hamiltonian Monte Carlo algorithms, see Step Size Adaptation in Hamiltonian Monte Carlo for more. This parameter will be tuned in the algorithm warmup but needs to start somewhere. I think Stan uses a value of 1 for this, so I follow them:
initial_step_size = [ tf.constant(INITIAL_STEP_SIZE, shape=[NUM_CHAINS] + [1] * (len(x.shape) - 1)) for x in initial_state ]
The shape magic is to allow each chain and each model parameter component to find a different step size in the warmup phase of the MCMC routine.
I'm going to skim over the last few steps now, as they are more-or-less boilerplate and hopefully self explanatory:
nuts = tfp.mcmc.NoUTurnSampler( target_log_prob_fn=target_log_prob_fn, step_size=initial_step_size, ) transformed_nuts = tfp.mcmc.TransformedTransitionKernel( inner_kernel=nuts, bijector=pinned_joint_bijector.bijectors, ) transformed_adaptive_nuts = tfp.mcmc.DualAveragingStepSizeAdaptation( inner_kernel=transformed_nuts, num_adaptation_steps=int(PROPORTION_ADAPTION_STEPS * NUM_BURNIN_STEPS), ) @tf.function(autograph=False, experimental_compile=True) def run_mcmc(): return tfp.mcmc.sample_chain( num_results=NUM_RESULTS, num_burnin_steps=NUM_BURNIN_STEPS, current_state=initial_state, kernel=transformed_adaptive_nuts, trace_fn=trace_fn, ) posterior_samples, mcmc_stats = run_mcmc()
And that's it! We have posterior samples and some mcmc diagnostics, but did it work? (words based on one of my favourite paper titles: Yes, but Did It Work?: Evaluating Variational Inference).
I think the first thing most people do after running some MCMC stuff is look at a
summary table and some plots, so lets do that. We use the same arviz.InferenceData
class as the prior predictive check section, but now we have more things to add in. This
is actually my least favourite step and would love to know a better way of doing this
(this is probably the arviz.from_tfp function but I couldn't get it to work).
Firstly, I don't want to have to remember the names and ordering of my model parameters defined in my joint distribution, so I resort to using a private member function to get the parameter names back:
parameter_names = pinned_joint_dist._flat_resolve_names() print(parameter_names)
['intercept', 'coefficients', 'observation_scale']
Secondly, we can sample from the posterior predictive distribution as easily as we could from the prior, this is one of my favourite things about TFP:
*_, posterior_predictive = joint_dist.sample(value=posterior_samples)
Thirdly, remember that arviz expects shapes to be of the form (chain, draw, *shape)
and TFP gives us (draw, chain, *shape)
hence we need to swap axes around.
trace = az.from_dict( prior={k: v[tf.newaxis, ...] for k, v in zip(parameter_names, prior_samples)}, posterior={ k: np.swapaxes(v, 0, 1) for k, v in zip(parameter_names, posterior_samples) }, prior_predictive={"observations": prior_predictive[tf.newaxis, ...]}, posterior_predictive={"observations": np.swapaxes(posterior_predictive, 0, 1)}, sample_stats={k: np.swapaxes(v, 0, 1) for k, v in mcmc_stats.items()}, coords={"observation": np.arange(N), "coefficient": np.arange(P)}, observed_data={"observations": data.observations}, dims={"observations": ["observation"], "coefficients": ["coefficient"]}, ) print(trace)
Inference data with groups: > posterior > posterior_predictive > sample_stats > prior > prior_predictive > observed_data
But now that is over - we can easily do lots of stuff:
print(az.summary(trace).filter(items=["mean", "hdi_3%", "hdi_97%", "ess_mean", "r_hat"]))
mean hdi_3% hdi_97% ess_mean r_hat intercept -0.226 -0.295 -0.155 13227.0 1.0 coefficients[0] -2.348 -2.416 -2.278 12830.0 1.0 coefficients[1] -0.228 -0.295 -0.158 13265.0 1.0 coefficients[2] 0.291 0.219 0.359 12464.0 1.0 coefficients[3] -0.020 -0.087 0.050 14511.0 1.0 coefficients[4] 0.385 0.318 0.457 13551.0 1.0 observation_scale 1.846 1.798 1.893 15892.0 1.0
I only filter the columns here to not clutter the post too much.
Again, the "true" values of the model parameters are:
for name in parameter_names: print(f"{name}: {getattr(data, name).numpy()}")
intercept: -0.20901992917060852 coefficients: [-2.3835316 -0.19555189 0.3054796 0.030938 0.39104107] observation_scale: 1.8365525007247925
so everything looks OK so far!
There are quite a few plots we can do, and I'm not going to talk about them in detail but will instead include hopefully helpful links:
We can do the posterior predictive check in the exact same way as the prior one - just
change the group
argument:
ax = az.plot_ppc(trace, group="posterior", num_pp_samples=500)
For more on this see A Conceptual Introduction to Hamiltonian Monte Carlo TLDR the closer the two densities look the more likely the algorithm has worked. The plot also shows the Bayesian fraction of missing information (BFMI), which is defined in the paper alongside this note:
Empirically, values of this energy Bayesian fraction of missing information below 0.3 have proven problematic, although more theoretical work is needed to formalize any exact threshold.
ax = az.plot_energy(trace)
axs = az.plot_dist_comparison(trace, var_names=["intercept"])
axs = az.plot_dist_comparison(trace, var_names=["observation_scale"])
Unfortunately due to the small uncertainty in comparison to the difference in locations this plot looks a bit rubbish, but often it's really useful so I've included it.
axs = az.plot_forest(trace, var_names=["coefficients"])
axs = az.plot_trace(trace, var_names=["intercept", "observation_scale"])
We've explored the posterior and everything looks fine, so next thing your pal comes along with a new design matrix with three rows and asks you to make predictions using all your fancy Bayesian stuff. Luckily it's easy to create a new joint distribution over these three new bits of data using the function we created at the start:
new_design_matrix = np.random.randn(3, P).astype(np.float32) new_joint_dist = make_joint_dist(new_design_matrix) print(new_joint_dist)
tfp.distributions.JointDistributionCoroutineAutoBatched("JointDistributionCoroutineAutoBatched", batch_shape=[], event_shape=StructTuple( intercept=[], coefficients=[5], observation_scale=[], observations=[3] ), dtype=StructTuple( intercept=float32, coefficients=float32, observation_scale=float32, observations=float32 ))
We can then sample observations using our posterior samples, that is, we are conditioning on the original dataset and predicting on this new one. This code is exactly the same as the first posterior predictive check we did:
*_, new_posterior_predictive = new_joint_dist.sample(value=posterior_samples)
We could give a point estimate as a prediction using e.g. the posterior mean by averaging/reducing the chain and draw dimensions:
print(tf.reduce_mean(new_posterior_predictive, axis=[0, 1]))
tf.Tensor([-0.7574589 -0.31100208 -3.9276493 ], shape=(3,), dtype=float32)
Or we could use arviz to plot the predictive distribution and admit our uncertainty:
new_trace = az.from_dict( posterior_predictive={"new_observations": np.swapaxes(new_posterior_predictive, 0, 1)}, coords={"new_observations": np.arange(3)}, dims={"new_observations": ["new_observation"]}, ) print(new_trace.posterior_predictive)
<xarray.Dataset> Dimensions: (chain: 8, draw: 1000, new_observation: 3) Coordinates: * chain (chain) int64 0 1 2 3 4 5 6 7 * draw (draw) int64 0 1 2 3 4 5 6 ... 993 994 995 996 997 998 999 * new_observation (new_observation) int64 0 1 2 Data variables: new_observations (chain, draw, new_observation) float32 -2.394 ... -1.649 Attributes: created_at: 2021-01-06T23:28:53.657267 arviz_version: 0.10.0
axs = az.plot_density( new_trace, group="posterior_predictive", outline=False, shade=0.8, point_estimate="mean", hdi_prob=1, )
This is quite a long post hopefully showing the beginnings of some sort of workflow with TFP and arviz. I'll try to keep it updated as I find out new things and hopefully have some time to focus on some parts in more detail or adding more steps (in particular model comparison).


Big number stats for what your entire morning revolves around. This is how the coffee reaches your cup.


This is a port of the classic game. It started on a slow work day, and is an experiment with HTML5 Canvas and WebSockets. But don't let that stop you from having fun!
Move using the arrow
keys.
Plant bombs using the ENTER
key.
Graphics by Twemoji
Textures from Minecraft


So you wanna play the GTA games to 100% and keep track of what you've done? Well, you've come to the right place.
If you want to add more feature, add more games or have an issue, check out the Github page.
lambdan.se


Pandora's Box
Welcome to Pandora's Box
View Passwords
Add Password
New Password
Delete Password
Logout


Select your size from S, M, L from left to right.


{%extends "nav_account.html"%} {%block title%}Elections{%endblock%} {%block content%}
You must be an active member of {{org_name}} in order to vote in this election, and you must have been it for four weeks before the opening of the election. To view the status of your membership, go to Your Membership.
{%endblock%}


|
Panda3D
|
This class is the main interface to controlling the render process. More...
Public Member Functions | |
| __init__ (Pipeline pipeline) | |
| Creates a new GraphicsEngine object. More... | |
| bool | addWindow (GraphicsOutput window, int sort) |
| This can be used to add a newly-created GraphicsOutput object (and its GSG) to the engine's list of windows, and requests that it be opened. More... | |
| dispatchCompute (const LVecBase3i work_groups, const ShaderAttrib sattr, GraphicsStateGuardian gsg) | |
| Asks the indicated GraphicsStateGuardian to dispatch the compute shader in the given ShaderAttrib using the given work group counts. More... | |
| bool | extractTextureData (Texture tex, GraphicsStateGuardian gsg) |
| Asks the indicated GraphicsStateGuardian to retrieve the texture memory image of the indicated texture and store it in the texture's ram_image field. More... | |
| flipFrame () | |
| Waits for all the threads that started drawing their last frame to finish drawing, and then flips all the windows. More... | |
| bool | getAutoFlip () |
| Returns the current setting for the auto-flip flag. More... | |
| Loader | getDefaultLoader () |
| Returns the Loader object that will be assigned to every GSG created with this GraphicsEngine. More... | |
| int | getNumWindows () |
| Returns the number of windows (or buffers) managed by the engine. More... | |
| bool | getPortalCull () |
| Returns the current setting for the portal culling flag. More... | |
| const ReMutex | getRenderLock () |
| Returns a ReMutex object that is held by the GraphicsEngine during the entire call to render_frame(). More... | |
| GraphicsThreadingModel | getThreadingModel () |
| Returns the threading model that will be applied to future objects. More... | |
| GraphicsOutput | getWindow (int n) |
| Returns the nth window or buffers managed by the engine, in sorted order. More... | |
| list | getWindows () |
| bool | isEmpty () |
| Returns true if there are no windows or buffers managed by the engine, false if there is at least one. More... | |
| GraphicsOutput | makeBuffer (GraphicsOutput host, str name, int sort, int x_size, int y_size) |
| Syntactic shorthand for make_output. More... | |
| GraphicsOutput | makeBuffer (GraphicsStateGuardian gsg, str name, int sort, int x_size, int y_size) |
| Syntactic shorthand for make_output. More... | |
| GraphicsOutput | makeOutput (GraphicsPipe pipe, str name, int sort, const FrameBufferProperties fb_prop, const WindowProperties win_prop, int flags, GraphicsStateGuardian gsg, GraphicsOutput host) |
| GraphicsOutput | makeParasite (GraphicsOutput host, str name, int sort, int x_size, int y_size) |
| Syntactic shorthand for make_buffer. More... | |
| openWindows () | |
| Fully opens (or closes) any windows that have recently been requested open or closed, without rendering any frames. More... | |
| readyFlip () | |
| Waits for all the threads that started drawing their last frame to finish drawing. More... | |
| removeAllWindows () | |
| Removes and closes all windows from the engine. More... | |
| bool | removeWindow (GraphicsOutput window) |
| Removes the indicated window or offscreen buffer from the set of windows that will be processed when render_frame() is called. More... | |
| renderFrame () | |
| Renders the next frame in all the registered windows, and flips all of the frame buffers. More... | |
| resetAllWindows (bool swapchain) | |
| Resets the framebuffer of the current window. More... | |
| setAutoFlip (bool auto_flip) | |
| Set this flag true to indicate the GraphicsEngine should automatically cause windows to sync and flip as soon as they have finished drawing, rather than waiting for all of the windows to finish drawing first so they can flip together. More... | |
| setDefaultLoader (Loader loader) | |
| Sets the Loader object that will be assigned to every GSG created with this GraphicsEngine. More... | |
| setPortalCull (bool value) | |
| Set this flag true to indicate the GraphicsEngine should start portal culling. More... | |
| setThreadingModel (const GraphicsThreadingModel threading_model) | |
| Specifies how future objects created via make_gsg(), make_buffer(), and make_window() will be threaded. More... | |
| syncFrame () | |
| Waits for all the threads that started drawing their last frame to finish drawing. More... | |
| Public Member Functions inherited from ReferenceCount | |
| int | getRefCount () |
| Returns the current reference count. More... | |
| ref () | |
| Explicitly increments the reference count. More... | |
| bool | testRefCountIntegrity () |
| Does some easy checks to make sure that the reference count isn't completely bogus. More... | |
| bool | testRefCountNonzero () |
| Does some easy checks to make sure that the reference count isn't zero, or completely bogus. More... | |
| bool | unref () |
| Explicitly decrements the reference count. More... | |
Static Public Member Functions | |
| static GraphicsEngine | getGlobalPtr () |
| Static Public Member Functions inherited from ReferenceCount | |
| static TypeHandle | getClassType () |
Public Attributes | |
| bool | auto_flip |
| Returns the current setting for the auto-flip flag. More... | |
| Loader | default_loader |
| Returns the Loader object that will be assigned to every GSG created with this GraphicsEngine. More... | |
| bool | portal_cull |
| Returns the current setting for the portal culling flag. More... | |
| const ReMutex | render_lock |
| Returns a ReMutex object that is held by the GraphicsEngine during the entire call to render_frame(). More... | |
| GraphicsThreadingModel | threading_model |
| Returns the threading model that will be applied to future objects. More... | |
| GraphicsOutput | windows [] |
| Returns the nth window or buffers managed by the engine, in sorted order. More... | |
| Public Attributes inherited from ReferenceCount | |
| int | ref_count |
| The current reference count. More... | |
This class is the main interface to controlling the render process.
There is typically only one GraphicsEngine in an application, and it synchronizes rendering to all all of the active windows; although it is possible to have multiple GraphicsEngine objects if multiple synchronicity groups are required.
The GraphicsEngine is responsible for managing the various cull and draw threads. The application simply calls engine->render_frame() and considers it done.
| __init__ | ( | Pipeline | pipeline | ) |
Creates a new GraphicsEngine object.
The Pipeline is normally left to default to NULL, which indicates the global render pipeline, but it may be any Pipeline you choose.
| bool addWindow | ( | GraphicsOutput | window, |
| int | sort | ||
| ) |
This can be used to add a newly-created GraphicsOutput object (and its GSG) to the engine's list of windows, and requests that it be opened.
This shouldn't be called by user code as make_output normally does this under the hood; it may be useful in esoteric cases in which a custom window object is used.
This can be called during the rendering loop, unlike make_output(); the window will be opened before the next frame begins rendering. Because it doesn't call open_windows(), however, it's not guaranteed that the window will succeed opening even if it returns true.
| dispatchCompute | ( | const LVecBase3i | work_groups, |
| const ShaderAttrib | sattr, | ||
| GraphicsStateGuardian | gsg | ||
| ) |
Asks the indicated GraphicsStateGuardian to dispatch the compute shader in the given ShaderAttrib using the given work group counts.
This can act as an interface for running a one-off compute shader, without having to store it in the scene graph using a ComputeNode.
Since this requires a round-trip to the draw thread, it may require waiting for the current thread to finish rendering if it is called in a multithreaded environment. However, you can call this several consecutive times on different textures for little additional cost.
The return value is true if the operation is successful, false otherwise.
| bool extractTextureData | ( | Texture | tex, |
| GraphicsStateGuardian | gsg | ||
| ) |
Asks the indicated GraphicsStateGuardian to retrieve the texture memory image of the indicated texture and store it in the texture's ram_image field.
The image can then be written to disk via Texture::write(), or otherwise manipulated on the CPU.
This is useful for retrieving the contents of a texture that has been somehow generated on the graphics card, instead of having been loaded the normal way via Texture::read() or Texture::load(). It is particularly useful for getting the data associated with a compressed texture image.
Since this requires a round-trip to the draw thread, it may require waiting for the current thread to finish rendering if it is called in a multithreaded environment. However, you can call this several consecutive times on different textures for little additional cost.
If the texture has not yet been loaded to the GSG in question, it will be loaded immediately.
The return value is true if the operation is successful, false otherwise.
| flipFrame | ( | ) |
Waits for all the threads that started drawing their last frame to finish drawing, and then flips all the windows.
It is not usually necessary to call this explicitly, unless you need to see the previous frame right away.
| bool getAutoFlip | ( | ) |
Returns the current setting for the auto-flip flag.
See set_auto_flip.
| Loader getDefaultLoader | ( | ) |
Returns the Loader object that will be assigned to every GSG created with this GraphicsEngine.
See GraphicsStateGuardian::set_loader().
|
static |
| int getNumWindows | ( | ) |
Returns the number of windows (or buffers) managed by the engine.
| bool getPortalCull | ( | ) |
Returns the current setting for the portal culling flag.
| const ReMutex getRenderLock | ( | ) |
Returns a ReMutex object that is held by the GraphicsEngine during the entire call to render_frame().
While you hold this lock you can be confident that no part of the frame will be rendered (at least by the app thread).
| GraphicsThreadingModel getThreadingModel | ( | ) |
Returns the threading model that will be applied to future objects.
See set_threading_model().
| GraphicsOutput getWindow | ( | int | n | ) |
Returns the nth window or buffers managed by the engine, in sorted order.
| list getWindows | ( | ) |
| bool isEmpty | ( | ) |
Returns true if there are no windows or buffers managed by the engine, false if there is at least one.
| GraphicsOutput makeBuffer | ( | GraphicsOutput | host, |
| str | name, | ||
| int | sort, | ||
| int | x_size, | ||
| int | y_size | ||
| ) |
Syntactic shorthand for make_output.
This is the preferred way to create an offscreen buffer, when you already have an onscreen window or another buffer to start with. For the first parameter, pass an existing GraphicsOutput object, e.g. the main window; this allows the buffer to adapt itself to that window's framebuffer properties, and allows maximum sharing of resources.
| GraphicsOutput makeBuffer | ( | GraphicsStateGuardian | gsg, |
| str | name, | ||
| int | sort, | ||
| int | x_size, | ||
| int | y_size | ||
| ) |
Syntactic shorthand for make_output.
This flavor accepts a GSG rather than a GraphicsOutput as the first parameter, which is too limiting and disallows the possibility of creating a ParasiteBuffer if the user's graphics hardware prefers that. It also attempts to request specific framebuffer properties and may therefore do a poorer job of sharing the GSG between the old buffer and the new.
For these reasons, this variant is a poor choice unless you are creating an offscreen buffer for the first time, without an onscreen window already in existence. If you already have an onscreen window, you should use the other flavor of make_buffer() instead, which accepts a GraphicsOutput as the first parameter.
| GraphicsOutput makeOutput | ( | GraphicsPipe | pipe, |
| str | name, | ||
| int | sort, | ||
| const FrameBufferProperties | fb_prop, | ||
| const WindowProperties | win_prop, | ||
| int | flags, | ||
| GraphicsStateGuardian | gsg, | ||
| GraphicsOutput | host | ||
| ) |
| GraphicsOutput makeParasite | ( | GraphicsOutput | host, |
| str | name, | ||
| int | sort, | ||
| int | x_size, | ||
| int | y_size | ||
| ) |
Syntactic shorthand for make_buffer.
| openWindows | ( | ) |
Fully opens (or closes) any windows that have recently been requested open or closed, without rendering any frames.
It is not necessary to call this explicitly, since windows will be automatically opened or closed when the next frame is rendered, but you may call this if you want your windows now without seeing a frame go by.
| readyFlip | ( | ) |
Waits for all the threads that started drawing their last frame to finish drawing.
Returns when all threads have actually finished drawing, as opposed to 'sync_frame' we seems to return once all draw calls have been submitted. Calling 'flip_frame' after this function should immediately cause a buffer flip. This function will only work in opengl right now, for all other graphics pipelines it will simply return immediately. In opengl it's a bit of a hack: it will attempt to read a single pixel from the frame buffer to force the graphics card to finish drawing before it returns
| removeAllWindows | ( | ) |
Removes and closes all windows from the engine.
This also cleans up and terminates any threads that have been started to service those windows.
| bool removeWindow | ( | GraphicsOutput | window | ) |
Removes the indicated window or offscreen buffer from the set of windows that will be processed when render_frame() is called.
This also closes the window if it is open, and removes the window from its GraphicsPipe, allowing the window to be destructed if there are no other references to it. (However, the window may not be actually closed until next frame, if it is controlled by a sub-thread.)
The return value is true if the window was removed, false if it was not found.
Unlike remove_all_windows(), this function does not terminate any of the threads that may have been started to service this window; they are left running (since you might open a new window later on these threads). If your intention is to clean up before shutting down, it is better to call remove_all_windows() then to call remove_window() one at a time.
| renderFrame | ( | ) |
Renders the next frame in all the registered windows, and flips all of the frame buffers.
| resetAllWindows | ( | bool | swapchain | ) |
Resets the framebuffer of the current window.
This is currently used by DirectX 8 only. It calls a reset_window function on each active window to release/create old/new framebuffer
| setAutoFlip | ( | bool | auto_flip | ) |
Set this flag true to indicate the GraphicsEngine should automatically cause windows to sync and flip as soon as they have finished drawing, rather than waiting for all of the windows to finish drawing first so they can flip together.
This only affects the timing of when the flip occurs. If this is true (the default), the flip occurs before render_frame() returns. If this is false, the flip occurs whenever flip_frame() is called, or at the beginning of the next call to render_frame(), if flip_frame() is never called.
| setDefaultLoader | ( | Loader | loader | ) |
Sets the Loader object that will be assigned to every GSG created with this GraphicsEngine.
See GraphicsStateGuardian::set_loader().
| setPortalCull | ( | bool | value | ) |
Set this flag true to indicate the GraphicsEngine should start portal culling.
| setThreadingModel | ( | const GraphicsThreadingModel | threading_model | ) |
Specifies how future objects created via make_gsg(), make_buffer(), and make_window() will be threaded.
This does not affect any already-created objects.
| syncFrame | ( | ) |
Waits for all the threads that started drawing their last frame to finish drawing.
The windows are not yet flipped when this returns; see also flip_frame(). It is not usually necessary to call this explicitly, unless you need to see the previous frame right away.
| bool auto_flip |
Returns the current setting for the auto-flip flag.
See set_auto_flip.
| Loader default_loader |
Returns the Loader object that will be assigned to every GSG created with this GraphicsEngine.
See GraphicsStateGuardian::set_loader().
| bool portal_cull |
Returns the current setting for the portal culling flag.
| const ReMutex render_lock |
Returns a ReMutex object that is held by the GraphicsEngine during the entire call to render_frame().
While you hold this lock you can be confident that no part of the frame will be rendered (at least by the app thread).
| GraphicsThreadingModel threading_model |
Returns the threading model that will be applied to future objects.
See set_threading_model().
| GraphicsOutput windows[] |
Returns the nth window or buffers managed by the engine, in sorted order.


| DOSE: |
|
IV |
Cocaine |
(powder / crystals) |
So I've been doing cocaine on a regular basis for about two years now. I guess I'm one of those addicts you'd call 'functioning' I go to a great university and I manage to keep friends and relationships and a steady job, etc... But non-the-less definitely an addict and I really have no intention to quit especially after the other night...
See I really like needles and I've done heroine a few times in the past and my friend S and I were sitting around with an 8 ball of coke. We had attempted to inject it before but had made the mistake of cooking it first.  I went first. I dropped the little rock-probably equivalent to a nice line-into the water and mixed it up with the small side of the plunger. It dissolved instantly, a sign of good coke. I put the needle back together and sucked the mixture into the shaft. I pulled the belt tight around my arm and plunged the needle into my vein. I pulled back and instantly saw blood. This caused my heart to race and I pushed the plunger forward.
Instant euphoria. I tasted the coke in my throat first, then noises were amplified, and then my body felt like it was floating - basically it felt like an orgasm but better. I was in heaven, I was God. I was warm happy and euphoric. My heart was racing and my blood was pounding, words can't really describe how amazing it was.
My friend S did her shot and instantly felt the same way. We continued to do the rest of the ball, commenting all the while about how shooting it was much better it was then snorting it. For one it doesn't clog my sinuses and it's a much cleaner high. Then after it was gone the crash came but then it was over. I typically get really depressed for a hour or so but shooting it makes the crash much more bearable. It just puts my body back to normal, no depression or anxiety, again it's amazing.
Shooting cocaine is literally the best feeling I've ever had followed by the least depressing crash I've ever had, so quitting is definitely not in my near future...
Experience Reports are the writings and opinions of the individual authors who submit them.
Some of the activities described are dangerous and/or illegal and none are recommended by Erowid.