Hi ,
You are invited to join the Luxe Talk Now, where you get to share your inspirations, from fresh styles, to great shoes, and your favorite grooming and beauty products.
When you join, there will be different activities to participate in each week. Whether it’s uploading a photo of your collection, commenting on the latest runway styles, telling us your favorite trends of the season, or taking a survey, they want to hear from you! Each time you participate in an activity, you will be entered to win $100!
Please click the “Join Now” button below to join now!


Assignment #2: TensorFlow, Neural Dependency Parsing, and RNN Language Modeling
Due Date: 2/8 (Thu) 11:59 PM PST.
Hard deadline: 2/11 (Sun) 11:59 PM PST with 3 late days.
In this assignment you will learn the fundamentals of TensorFlow, use TensorFlow to implemented a feed-forward neural network for transition-based dependency parsing, and delve into backpropagation through time by computing the gradients for a recurrent neural network language model.
Solutions
Written solutions have been posted here.Setup
Note: Please be sure you have Python 2.7.x installed on your system. The following instructions should work on Mac or Linux. If you have any trouble getting set up, please come to office hours and the TAs will be happy to help.
Get the code: Download the starter code here and the assignment handout here.
Python package requirements: The core requirements for this assignment are
- tensorflow
- numpy
See the TensorFlow webpage for installation directions. You should use the latest version of TensorFlow (1.4.1).
Submitting your work
Submission instructions are posted on Piazza here.Assignment Overview (Tasks)
There will be three parts to this assignment. Coding the neural dependency parsing (Q2 part (h)) is designed to be completed after Part 1. The rest of the assignment can be done independently. We recommend reading the assignment carefully and starting early as some parts may take significant time.


Deep Art fun
January 16, 2016
Last weekend I finally took some of personal photos and artwork to "paint" them in, but not limited to, Van Gogh style. The "painting" part was actually done by DeepArt algorithm - an inspiring application from Deep learning researchIt’s worth to note that Artomatix, Inc. - an Irish startup - had introduced a similar work several months before DeepArt algorithm was published. . In short, the Deep learning model - a deep neural network - takes your photo (referred as content-image henceforth) and a reference style (combination of 1 or more style-images) as the inputs and produces your photo in that style.
Demo
I used jcjohnson’s implementation of DeepArt algorithm for this demo.
Here we have the content-image on the top-left, the style-image is an artwork by artist Leonid Afremov on the top-right, and the photo generated in Afremov's style at the bottom.
The machine did not “paint” the picture out of the blue, it's an iterative learning process. In short, it starts with a blank (noisy) canvas and tries to learn prominent features in both content-image and style-image(s) and present them on the final product. That learning process is mathematically expressed as optimization process of a function representing the combined similarity between the output and the inputs. Optimizing such function requires an algorithm which goes through series of updating iterations, visualized in the following gif.
More images
Note: Due to hardware constrainst, most of images below were generated from non-optimal configs, thus did not yield their best output. I will be updating results from optimal configs soon enough.
Style image: creativemints's
produced by NIN model (non-optimal config)
produced by VGG-19 model (more optimal config)
(Old images) Technical bit: Old images were produced by NIN model + Adam optimization. If hardware constrainst is not a problem, visual results should be considerably improved by using VGG-19 model + L-BFGS optimization.
Style image: Leonid Afremov's
Style image: Van Gogh's Starry Night
Style image: Van Gogh's Starry Night
Style image: Van Gogh's Wheat Field with Crows
Observation
- Size matters. The higher the output image resolution, the more details in the content-image can be explicitly expressed by the indicated style. However, memory is a big trade-off. Default output image size (max. 512px-wide) on an NIN took as small as 5-600MB, but a 784px image devours 3 times as much!
- For visual appeal, (i) an expressive style and/or (ii) resemblance between content-Style image pair is the key. "Expressive" here is not only about abstract or surreal in specific, but about texture i.e. discernible patterns in colors, strokes, blobs in general. Example work: Leonid Afremov's painting, The Scream, Monet's water lilies, etc. Eventually, using style-images referenced from complicated realism pieces by the old masters, e.g. Jean Leon Gerome, usually yields unsatisfiying results. Secondly, although not always, content- and style-images of similar semantics pair well with each other. For example, a landscape picture should play along well with Starry Night or Wheat Field and the Crows; a portrait and one of Picasso's self-portraits also make a nice combination.
- I drool over a Titan X.
Hardware setup
The main work-horse is a humble 2GB nVidia GTX 660 GPUi.e. common video graphic card for gaming plugged in a dated LGA 775 desktop. Each image was generated in ~90 seconds (for a 512px-image) or 2-3 mins (for a 784px-image)Recent advances have largely reduce generation time down to just a few seconds . GPU(s) with generous memory is desirable.
It's much faster to run DeepArt algorithm on GPU(s) instead of common CPUs. Even a modern i7 K-series can take upto hour to train and generate 1 image of above sizes. However, running on CPU can take advantages of abundant host memory (easily >8GB on a single machine), not to mention the possibility to be accompanied with a coprocessor (eg: Xeon Phi), or running on distributed platform.
References
A Neural Algorithm of Artistic Style (Gatys et al, 2015) i.e. DeepArt algorithm


---
layout: post
title: "How to prepare for the qulifying"
date: 2016-03-31
comments: true
archive: true
notes: true
tags: notes
excerpt: Thoughts and notes regarding preparation of qualifying exam
---
Qualifying
Coming up with broad topics that span the literature
Putting together a list of about 30 readings per topic, broken down into subtopics
Coming up with 2-3 proposed questions on each topic


eeg_evoked()
class introduced to hold ERPSeeg_average()
function to calculate averages (e.g. ERPs) from eeg_epochs
objectsas.data.frame.eeg_evoked()
introduced to handle conversion of eeg_evoked objects to data frames.compute_psd()
function development, converted to S3method.topoplot()
properly checks for existing chan_info in eeg_data
objectsplot_timecourse()
and plot_butterfly()
modified to deal with eeg_evoked
objects.eeg_downsample()
function added to downsample EEG data by an integer factor.tag_events()
function added to give labels to event codes.list_events()
added to display unique event codes and their associated labels.select_epochs()
now allows selection of epochs by event code or event label.erp_raster()
- plot ERPs across the scalp as an ERP imageeeg_combine()
- combine multiple eeg_data
or eeg_epochs
objects into oneeeg_epochs()
now also handles downsampled data appropriately.select_times()
no longer leaves “epoch” column in eeg_epochs
objects.topoplot()
now calls a separate function (gam_topo()
) to create GAM smoothsbrowse_data()
major speed-ups, no longer converts to long format until necessary. Converted to S3method.interactive_scalp()
fixed plotting of individual electrodesinterp_elecs()
function to perform spherical spline interpolation of individual electrodeseeg_ar_thresh()
simple absolute value thresholding addedplot_electrodes()
Produces a 2D or interactive 3D plot of electrode locationsiir_filt()
now also filters reference channelsload_set()
command added to load EEGLAB .set filesselect_times()
to an S3 generic methodselect_times.eeg_data
select_times.eeg_epochs
iir_filt()
to an S3 generic methodiir_filt.eeg_data
iir_filt.eeg_epochs


404 Not Found
Have you seen this man?
Please report your sighting to<EMAIL_ADDRESS>--- title: 404 --- @@include('../templates/_head.tpl')
@@include('../templates/_homenav.tpl')Have you seen this man?
Please report your sighting to members@colab.coop.


DB Monitor Configuration
Getting started with DB Monitor is simple!
When first first launched you should see a window that looks like this:
If you don't see one, don't panic. Simple select File->New from the Application Menu:
To Edit the configuration, simply hit Command-, or select the Document->Configure Pools option from the Application Menu:
This will bring up the configuration pane, which will look like this:
Our first step will be to create a new pool to hold a group of servers. On the left hand side of the panel, you should see a Pools list. Click the plus sign in the bottom right hand corner to add a new pool:
This will add a new pool to be edited to your configuration which should be automatically selected by default:
You can name the pool by simply typing into the column on the left OR by typing into the text entry box just to it's right.
A pool of servers is a group of hosts that perform a similar function, and use the same username/password to log into them. Your cluster may only have one master and one group of slaves, and you may only need a single pool to describe your infrastructure. Or you may have several projects, each with its own master and set of slaves. I find it is good to group systems into pools by master, myself. You may make pools for dev, qa, staging, production, etc - or you can separate them into individual documents so you can have multiple windows open showing different clusters or systems, it's entirely up to you!
Let's go ahead and enter a username and password combination to log into this pool of systems now:
Next as you can see it has automatically filled out two numbers. The first is 'Show Query After' and the second is 'Track Query After'.
The first number tells the application how many seconds to wait for a query to run before you want to display it in the event log. Showing a query doesn't mean that you necessarily want to track more information about that query, only that you are interested in seeing it displayed in the log at runtime (default value is 1 second).
The second number tells the application how many seconds is too long for a query to run. Once a running query has been running for that amount of time (default value is 5 seconds) it highlights the query in red in the event log. If you have Query Tracking enabled it will also leave that query in the event log, even after it has stopped running for inspection later. We'll look at Query Tracking more when we go over the Event Log Viewer.
Now we're ready to add some hosts to our pool! Start by clicking the plus sign at the bottom right hand corner of the lower table:
Once you've done that, just select the text area under Hostname and start typing. It will fill out the Port column to 3306 by default for MySQL, but you can edit that also.
Rinse and repeat until you've added all of the hosts that you want to put in this pool:
But wait! If we were to hit the Done button now, nothing would show up in our event logs, because we've set no pools or hosts to active! Setting hosts or pools to Active/Inactive is a very useful way to filter out host information that you aren't interested in if you need to quickly drill down into a host or pool in order to get a better idea of what is going on. It's also a good way to filter OUT a spammy host that is having problems. Let's check off Active for all of the hosts we've entered now:
We did put some replicated slaves in the list, so we should probably check them off as slaves also so we gather slave statistics:
But we've skipped a whole column! There's an Alias column that you can put a text string into if your host names are very long or you need to put a FQDN in so that your hosts resolve properly. Or, if you have several MySQL servers that are running off different ports (or you're doing port forwarding across an SSH tunnel). Let's pretend that my hostnames need FQDN so I'm going to enter some short names for them. Unlike host names, I can put as many spaces in the Aliases as I'd like:
Now we're done configuring our first pool! Simply hit the Done button in order to return to the event Screen!
You'll know you've probably entered something wrong if you immediately see a screen like this once you exit out, make sure you've entered your username and password correctly!
And make sure to save your document by hitting Command-S or selecting File->Save from the Application Menu!


--- layout: default title: "404: Page not found" permalink: 404.html ---
Sorry, I've misplaced the URL or it points to something that doesn't exist. I really don't know how you got here. Go back home and try to find it again.


Angry
Happy
Sad
Calm
Suggestions For You
We Think You Will Like This!
A look back into the past...


public final class OtpTools
extends java.lang.Object
| Modifier and Type | Class and Description |
|---|---|
static class |
OtpTools.FormatType
Supported formatting modes for OTP.
|
static class |
OtpTools.ScrambleType
Supported scramble modes for OTP.
|
| Modifier and Type | Method and Description |
|---|---|
static SecureString |
formatOtp(OtpTools.FormatType mode,
SecureString otp)
Format the OTP with according provided mode.
|
static SecureString |
padOtpWithZero(SecureString otp,
int minLength)
left pad an OTP with Zero if needed.
|
static SecureString |
scrambleOtp(OtpTools.ScrambleType mode,
SecureString otp)
Compute a scrambled OTP from original OTP.
|
public static SecureString scrambleOtp(OtpTools.ScrambleType mode, SecureString otp)
mode
- - Specifies the mode of scrambling functionotp
- - Original otp. Length must greater or equal to 5 and only
digits are accepted.java.lang.IllegalArgumentException
- When scramble type or otp is invalidpublic static SecureString formatOtp(OtpTools.FormatType mode, SecureString otp)
mode
- - Specifies the mode of formatting functionotp
- - original OTP
if mode is see OtpTools.FormatType.CAP
| OTP length | Formatted OTP |
| 1 | X |
| 2 | XX |
| 3 | XXX |
| 4 | XXXX |
| 5 | XX XXX |
| 6 | XXX XXX |
| 7 | XXX XXXX |
| 8 | XXXX XXXX |
| 9 | XXX XXX XXX |
| 10 | XX XXXX XXXX |
| 11 | XXXXXXXXXXX |
| 12 | XXXXXXXXXXXX |
| >12 | not changed |
java.lang.IllegalArgumentException
- when the otp or format type is invalidjava.lang.IllegalArgumentException
- when the mode is CAP
and
the otp contains a non-digit character.public static SecureString padOtpWithZero(SecureString otp, int minLength)
otp
- OTP to pad.minLength
- The minimum length of the padded OTP. If minLength is lower than the otp length, no padding is added and a copy of otp is returned (so original one can be wiped).


My name is Morgan and I am a west coast theatre nerd living in the City of Brotherly Love! By day I am an Administrative Coordinator for a large performing arts center, and by night I am a Full Stack Web Developer. Proficient in HTML5, CSS, Javascript, JQuery, front end CSS frameworks (Bootstrap, Foundation), Databases (MySql, MongoDB, IndexedDB, Sequelize), Node, NPM, and Express.
Resume | GitHub | LinkedIn


Project 1: Visual Message to Celestial Beings
This plaque is an SOS signal to any alien(s) that happen to come across it. Earth is a gosh darn mess, with many terrible things happening at once. It is undeniable that we need help. I propose that we hire extraterrestrial help.
I included the diagram seen in both the pioneer plaque and the voyager record because it's purpose is to pinpoint our location. I have also included a diagram of our solar system and an arrow to indicate exactly which planet in said solar system we are. I included Pluto in said diagram because even if Pluto isn't a planet it is still a round planet-like thing that orbits around our sun, and therefore it deserves to feel included. I added a picture of our planet so when someone finally finds our planet they can visually identify it.
One of the bigger challemnges in this project was to adequately communicate that we need help without the use of words, as aliens probably do not speak English (or any human language for that matter). I decided to include icons in the shape of triangles because organisms (even if extraterrestrial) recognize that objects with sharp corners are either dangerous or in danger. I have also included "SOS" in morse code in hopes that aliens understand morse code.


Welcome to Construct, a visual layout editor based on Blueprint & jQuery! This is version 1.0, finalized on March 5, 2010 and released for public consumption under the FreeBSD License. This project was built by Christian Montoya, and exists both as a useful tool for CSS designers and as proof that a visual layout editor is possible to acheive with clean CSS & semantic HTML. Read on for an explanation of the controls.
Here are the keyboard mappings for Construct:
Also, you can click on any container or column to select it.
Construct is a visual layout editor based on Blueprint.
It was inspired by the Blueprint Layout Builder.
It is powered by jQuery.
And, finally, it was written by Christian Montoya.


labeled
by None at 2022-11-15 22:16:52.labeled
by None at 2022-11-15 22:16:52.assigned
by Jane Sandberg at 2022-11-15 23:45:21.referenced
by None at 2022-11-16 15:43:41.merged
by None at 2022-11-16 15:43:41.closed
by None at 2022-11-16 15:43:41.head_ref_deleted
by None at 2022-11-16 15:43:42.deployed
by None at 2022-11-16 15:39:27.referenced
by Shaun Ellis at 2022-11-16 17:08:18.merged
by Shaun Ellis at 2022-11-16 17:08:18.closed
by Shaun Ellis at 2022-11-16 17:08:18.head_ref_deleted
by Shaun Ellis at 2022-11-16 17:08:19.closed
by Shaun Ellis at 2022-11-16 17:08:20.closed
by Christina Chortaria at 2022-11-16 15:30:30.head_ref_force_pushed
by Christina Chortaria at 2022-11-16 15:11:05.ready_for_review
by Christina Chortaria at 2022-11-16 15:21:05.deployed
by None at 2022-11-16 15:21:24.referenced
by Christina Chortaria at 2022-11-16 15:30:28.merged
by Christina Chortaria at 2022-11-16 15:30:28.closed
by Christina Chortaria at 2022-11-16 15:30:29.head_ref_deleted
by Christina Chortaria at 2022-11-16 15:30:29.@regineheberlein what are the delivery locations for each one of these five locations?
@christinach all of them are non-requestable from the catalog. thanks for checking!
assigned
by Christina Chortaria at 2022-11-16 01:01:51.mentioned
by None at 2022-11-16 01:15:02.subscribed
by None at 2022-11-16 01:15:02.mentioned
by Christina Chortaria at 2022-11-16 14:36:52.subscribed
by Christina Chortaria at 2022-11-16 14:36:52.convert_to_draft
by Christina Chortaria at 2022-11-16 15:32:29.head_ref_force_pushed
by Christina Chortaria at 2022-11-16 15:33:11.renamed
by Christina Chortaria at 2022-11-16 15:35:58.ready_for_review
by Christina Chortaria at 2022-11-16 20:36:42.convert_to_draft
by Christina Chortaria at 2022-11-16 20:36:50.head_ref_force_pushed
by Christina Chortaria at 2022-11-16 23:28:58.head_ref_force_pushed
by Christina Chortaria at 2022-11-16 23:32:51.head_ref_force_pushed
by Christina Chortaria at 2022-11-17 01:47:21.head_ref_force_pushed
by Christina Chortaria at 2022-11-17 01:56:47.head_ref_force_pushed
by Christina Chortaria at 2022-11-17 02:23:41.head_ref_force_pushed
by Christina Chortaria at 2022-11-17 02:29:04.referenced
by None at 2022-11-16 15:45:10.merged
by None at 2022-11-16 15:45:10.closed
by None at 2022-11-16 15:45:10.deployed
by None at 2022-11-15 23:30:00.renamed
by Jane Sandberg at 2022-11-15 23:34:14.It's there. https://dpul.princeton.edu/cotsen-collection/catalog/8g84mw60f
closed
by None at 2022-11-16 16:03:48.I wrote a ticket for documenting how to fix this: https://github.com/pulibrary/rdss-handbook/issues/95
closed
by Bess Sadler at 2022-11-16 15:04:14.assigned
by Kate Lynch at 2022-11-16 15:35:22.@hackartisan I simplified the mosaic method, added comments, and fixed some logic that was incorrect in our original implementation. A "mosaic" should have two or more raster files.
mentioned
by None at 2022-11-16 18:19:35.subscribed
by None at 2022-11-16 18:19:35.review_requested
by Eliot Jordan at 2022-11-16 18:19:42.deployed
by None at 2022-11-16 18:20:05.head_ref_force_pushed
by Eliot Jordan at 2022-11-16 20:47:28.review_requested
by Eliot Jordan at 2022-11-16 20:48:56.referenced
by Eliot Jordan at 2022-11-16 21:24:56.merged
by Eliot Jordan at 2022-11-16 21:24:56.closed
by Eliot Jordan at 2022-11-16 21:24:56.head_ref_deleted
by Eliot Jordan at 2022-11-16 21:24:57.assigned
by Max Kadel at 2022-11-16 14:47:04.head_ref_force_pushed
by Jane Sandberg at 2022-11-16 00:26:10.deployed
by None at 2022-11-16 00:26:56.head_ref_force_pushed
by Jane Sandberg at 2022-11-16 00:28:29.head_ref_force_pushed
by Jane Sandberg at 2022-11-16 00:34:38.deployed
by None at 2022-11-16 00:34:42.deployed
by None at 2022-11-16 16:10:47.referenced
by Christina Chortaria at 2022-11-16 16:54:52.merged
by Christina Chortaria at 2022-11-16 16:54:52.closed
by Christina Chortaria at 2022-11-16 16:54:52.head_ref_deleted
by Christina Chortaria at 2022-11-16 16:54:53.assigned
by Jane Sandberg at 2022-11-16 00:23:25.closed
by Christina Chortaria at 2022-11-16 16:54:53.Github glitch - was already merged.
closed
by Max Kadel at 2022-11-16 18:42:46.head_ref_deleted
by Max Kadel at 2022-11-16 18:42:49.renamed
by Christina Chortaria at 2022-11-16 19:42:10.labeled
by Christina Chortaria at 2022-11-16 19:43:07.labeled
by None at 2022-11-16 16:55:52.labeled
by None at 2022-11-16 16:55:52.deployed
by None at 2022-11-16 18:43:14.referenced
by Christina Chortaria at 2022-11-16 19:58:35.merged
by Christina Chortaria at 2022-11-16 19:58:36.closed
by Christina Chortaria at 2022-11-16 19:58:36.head_ref_deleted
by Christina Chortaria at 2022-11-16 19:58:36.labeled
by None at 2022-11-16 04:57:59.labeled
by None at 2022-11-16 04:57:59.Superseded by #87.
closed
by None at 2022-11-16 04:58:02.head_ref_deleted
by None at 2022-11-16 04:58:04.Notes from All-hands session which may be relevant: https://docs.google.com/document/d/1C_VeXTF7-9zImLq6j_0TJWtumSehDuJ4-svgISQRBlo/edit#heading=h.wsvjvyxj8wkh
Proposal: A ruby script that uses the github API to fetch security / dependabot alerts for all (or a subset of) PUL projects. For each alert, it creates a github issue in https://github.com/PrincetonUniversityLibrary/security/issues, prefixed with some kind of persistent ID. The issue holds a link to the dependabot info, and a list of all affected PUL apps. The issue serves as a central location for developers to discuss approaches to resolution.
The script runs weekly. It doesn't remove existing issues but at each run it ensures the list of affected apps is current.
Coverage remained the same at 94.496% when pulling b112ebbcc3ff34182ba4fd4f1e1983b10c05a662 on revert-3256-illiad_msg into 4afec3c74b5a1512cce25258d6267838c22b38c0 on main.
referenced
by Kevin Reiss at 2022-11-16 16:26:14.merged
by Kevin Reiss at 2022-11-16 16:26:14.closed
by Kevin Reiss at 2022-11-16 16:26:15.head_ref_deleted
by Kevin Reiss at 2022-11-16 16:26:15.assigned
by Max Kadel at 2022-11-16 18:03:48.referenced
by Max Kadel at 2022-11-16 18:24:51.Coverage increased (+0.02%) to 94.514% when pulling f1153429076ce5a510c8772a139a19973078c9da on i3253smallersitemap_buckets into b43c6bee9c46bd0369afd4a74fb09fc22d57a469 on main.
Deployed to catalog-staging with new configuration & cleared the Rails cache, and the sitemap links are all 4 characters
deployed
by None at 2022-11-16 18:34:22.referenced
by Max Kadel at 2022-11-16 18:43:48.merged
by Max Kadel at 2022-11-16 18:43:48.closed
by Max Kadel at 2022-11-16 18:43:48.head_ref_deleted
by Max Kadel at 2022-11-16 18:43:49.labeled
by Jane Sandberg at 2022-11-16 15:55:28.closed
by Max Kadel at 2022-11-16 18:43:49.Coverage remained the same at 94.496% when pulling d2a3111c6e3b4c968f991acc7e5fa72b9ae66186 on 2696-availabilityjs into 7eb6dae3adfd10f59fc694b7369d5f465cf338c7 on main.
ready_for_review
by Christina Chortaria at 2022-11-16 15:42:14.head_ref_force_pushed
by Christina Chortaria at 2022-11-16 18:30:41.head_ref_force_pushed
by Christina Chortaria at 2022-11-16 18:51:50.deployed
by None at 2022-11-16 19:11:23.referenced
by Kevin Reiss at 2022-11-16 20:53:17.merged
by Kevin Reiss at 2022-11-16 20:53:17.closed
by Kevin Reiss at 2022-11-16 20:53:17.head_ref_deleted
by Kevin Reiss at 2022-11-16 20:53:18.closed
by Kevin Reiss at 2022-11-16 20:53:18.Coverage increased (+0.02%) to 94.514% when pulling 1f13118049a1cd2688fbab521560a4d7297c82b8 on dependabot/npmandyarn/loader-utils-1.4.2 into a000e50445b357172950603d4cfe31905d035b62 on main.
labeled
by None at 2022-11-16 20:54:34.labeled
by None at 2022-11-16 20:54:34.deployed
by None at 2022-11-17 00:05:56.referenced
by Jane Sandberg at 2022-11-17 00:16:56.merged
by Jane Sandberg at 2022-11-17 00:16:56.closed
by Jane Sandberg at 2022-11-17 00:16:56.head_ref_deleted
by Jane Sandberg at 2022-11-17 00:16:56.Coverage remained the same at 94.496% when pulling 26249ec6e097e90db1f48dccc736fd0de556ea4a on 3248-clancy-error-details into 7eb6dae3adfd10f59fc694b7369d5f465cf338c7 on main.
ready_for_review
by None at 2022-11-16 20:50:32.referenced
by Jane Sandberg at 2022-11-17 00:21:33.merged
by Jane Sandberg at 2022-11-17 00:21:33.closed
by Jane Sandberg at 2022-11-17 00:21:33.head_ref_deleted
by Jane Sandberg at 2022-11-17 00:21:34.closed
by Jane Sandberg at 2022-11-17 00:21:34.Coverage increased (+0.002%) to 94.498% when pulling 48003c4a0481dfc6a6e40f19d4f4aab15237b22c on usestackmapfor_firestone into 7eb6dae3adfd10f59fc694b7369d5f465cf338c7 on main.
@maxkadel when we deploy this to a branch does it default to "on" or "off"?
deployed
by None at 2022-11-16 18:47:48.ready_for_review
by Max Kadel at 2022-11-16 19:17:05.mentioned
by Max Kadel at 2022-11-17 00:41:58.subscribed
by Max Kadel at 2022-11-17 00:41:59.head_ref_force_pushed
by Hector Correa at 2022-11-16 13:56:09.referenced
by Bess Sadler at 2022-11-16 15:17:31.merged
by Bess Sadler at 2022-11-16 15:17:31.closed
by Bess Sadler at 2022-11-16 15:17:31.head_ref_deleted
by Bess Sadler at 2022-11-16 15:17:32.closed
by Bess Sadler at 2022-11-16 15:17:32.unassigned
by None at 2022-11-16 15:22:17.ready_for_review
by Tyler Wade at 2022-11-16 17:49:49.referenced
by Bess Sadler at 2022-11-16 21:59:58.merged
by Bess Sadler at 2022-11-16 21:59:58.closed
by Bess Sadler at 2022-11-16 21:59:58.head_ref_deleted
by Bess Sadler at 2022-11-16 21:59:59.referenced
by Bess Sadler at 2022-11-16 22:00:21.merged
by Bess Sadler at 2022-11-16 22:00:21.closed
by Bess Sadler at 2022-11-16 22:00:21.head_ref_deleted
by Bess Sadler at 2022-11-16 22:00:22.referenced
by Bess Sadler at 2022-11-16 22:00:54.merged
by Bess Sadler at 2022-11-16 22:00:54.closed
by Bess Sadler at 2022-11-16 22:00:54.head_ref_deleted
by Bess Sadler at 2022-11-16 22:00:55.We need a design decision about how we will handle migrating works that do not have a doi.
labeled
by Tyler Wade at 2022-11-16 15:54:47.assigned
by Bess Sadler at 2022-11-16 15:54:54.unlabeled
by Bess Sadler at 2022-11-16 22:04:23.closed
by Bess Sadler at 2022-11-16 22:04:34.referenced
by Bess Sadler at 2022-11-16 22:08:32.merged
by Bess Sadler at 2022-11-16 22:08:32.closed
by Bess Sadler at 2022-11-16 22:08:32.head_ref_deleted
by Bess Sadler at 2022-11-16 22:08:33.<?xml version="1.0"?>
<resource xsi:schemaLocation='http://datacite.org/schema/kernel-4 http://schema.datacite.org/meta/kernel-4/metadata.xsd' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xmlns='http://datacite.org/schema/kernel-4'>
<identifier identifierType='DOI'>10.34770/7hyr-rf67</identifier>
<creators>
<creator>
<creatorName>Ghannam, Khaled</creatorName>
<givenName>Khaled</givenName>
<familyName>Ghannam</familyName>
</creator>
<creator>
<creatorName>Poggi, Davide</creatorName>
<givenName>Davide</givenName>
<familyName>Poggi</familyName>
</creator>
<creator>
<creatorName>Katul, Gabriel</creatorName>
<givenName>Gabriel</givenName>
<familyName>Katul</familyName>
</creator>
<creator>
<creatorName>Bou-Zeid, Elie</creatorName>
<givenName>Elie</givenName>
<familyName>Bou-Zeid</familyName>
</creator>
</creators>
<titles>
<title>
Data from a flume experiment of passive scalar diffusion within vegetation
canopies using laser-induced fluorescence
</title>
</titles>
<publisher>Princeton University</publisher>
<resourceType resourceTypeGeneral='Dataset'/>
<publicationYear>2020</publicationYear>
<relatedIdentifiers>
<relatedIdentifier relationType='IsIdenticalTo' relatedIdentifierType='ARK'>ark:/88435/dsp01qj72pb044</relatedIdentifier>
</relatedIdentifiers>
<version>1</version>
<rightsList>
<rights rightsURI='https://creativecommons.org/licenses/by/4.0/' rightsIdentifier='CC BY'>Creative Commons Attribution 4.0 International</rights>
</rightsList>
<descriptions>
<description descriptionType='Other'>
This dataset is a sequence of laser-induced fluorescence images of a dye
injected in a channel flow with canopy-like stainless steel rods
simulating a vegetation canopy stand. The data is acquired close to the
channel bottom at z/h=0.2, where z is the height referenced to the channel
bed and h is the canopy height. The dataset provides spatial distribution
of scalar concentration in a plane parallel to the channel bed. The data
has been used (but the data itself has not been published or available to
the public) in previous work. The references are: Ghannam, K., Poggi, D.,
Porporato, A., & Katul, G. (2015). The spatio-temporal statistical
structure and ergodic behaviour of scalar turbulence within a rod canopy.
Boundary-Layer Meteorology,157(3), 447–460. Ghannam, K, Poggi, D.,
Bou-Zeid, E., Katul, G. (2020). Inverse cascade evidenced by information
entropy of passive scalars in submerged canopy flows. Geophysical Research
Letters (accepted). The attached readme.txt file explains the data
attributes
</description>
</descriptions>
</resource>
closed
by Bess Sadler at 2022-11-16 22:09:26.After investigation I decided to abandon this effort. It looks like schema validation happens in the datacite-mapping gem and I was unable to create an invalid object. No need for us to do this schema validation ourselves unless we want to impose different requirements on our records than DataCite does.
closed
by Bess Sadler at 2022-11-16 21:51:12.reopened
by Bess Sadler at 2022-11-16 22:41:05.referenced
by Bess Sadler at 2022-11-16 15:18:35.merged
by Bess Sadler at 2022-11-16 15:18:35.closed
by Bess Sadler at 2022-11-16 15:18:35.head_ref_deleted
by Bess Sadler at 2022-11-16 15:18:36.closed
by Francis Kayiwa at 2022-11-15 22:48:23.referenced
by None at 2022-11-16 15:58:28.merged
by None at 2022-11-16 15:58:28.closed
by None at 2022-11-16 15:58:28.head_ref_deleted
by None at 2022-11-16 15:58:29.referenced
by Rebecca Sutton Koeser at 2022-11-16 19:40:54.merged
by Rebecca Sutton Koeser at 2022-11-16 19:40:54.closed
by Rebecca Sutton Koeser at 2022-11-16 19:40:54.head_ref_deleted
by Rebecca Sutton Koeser at 2022-11-16 19:40:55.head_ref_force_pushed
by Jane Sandberg at 2022-11-15 22:50:27.head_ref_force_pushed
by Jane Sandberg at 2022-11-15 23:00:08.ready_for_review
by Jane Sandberg at 2022-11-15 23:32:34.referenced
by Francis Kayiwa at 2022-11-16 21:46:35.merged
by Francis Kayiwa at 2022-11-16 21:46:35.closed
by Francis Kayiwa at 2022-11-16 21:46:35.head_ref_deleted
by Francis Kayiwa at 2022-11-16 21:46:36.closed
by Francis Kayiwa at 2022-11-16 21:57:32.closed
by Francis Kayiwa at 2022-11-16 21:58:20.head_ref_deleted
by Kevin Reiss at 2022-11-16 13:13:45.head_ref_deleted
by Kevin Reiss at 2022-11-16 13:13:47.head_ref_deleted
by Kevin Reiss at 2022-11-16 13:13:48.head_ref_deleted
by Kevin Reiss at 2022-11-16 13:14:00.head_ref_deleted
by Kevin Reiss at 2022-11-16 13:14:01.head_ref_deleted
by Kevin Reiss at 2022-11-16 13:14:02.head_ref_deleted
by Kevin Reiss at 2022-11-16 13:14:06.head_ref_restored
by Kevin Reiss at 2022-11-16 13:14:48.labeled
by Kevin Reiss at 2022-11-16 21:58:49.renamed
by Jane Sandberg at 2022-11-15 23:24:30.deployed
by None at 2022-11-16 22:13:22.@tpendragon thanks for catching that. How does this look?
mentioned
by Trey Pendragon at 2022-11-16 15:57:20.subscribed
by Trey Pendragon at 2022-11-16 15:57:20.referenced
by Trey Pendragon at 2022-11-16 17:59:14.merged
by Trey Pendragon at 2022-11-16 17:59:14.closed
by Trey Pendragon at 2022-11-16 17:59:14.head_ref_deleted
by Trey Pendragon at 2022-11-16 17:59:15.closed
by Trey Pendragon at 2022-11-16 17:59:16.labeled
by None at 2022-11-16 21:37:51.labeled
by None at 2022-11-16 21:37:51.Superseded by #1127.
closed
by None at 2022-11-16 21:37:53.head_ref_deleted
by None at 2022-11-16 21:37:55.transferred
by Chuck McCallum at 2022-11-16 23:18:51.closed
by Bess Sadler at 2022-11-16 15:22:52.labeled
by None at 2022-11-16 23:46:01.labeled
by None at 2022-11-16 23:46:01.Superseded by #214.
closed
by None at 2022-11-16 23:46:04.head_ref_deleted
by None at 2022-11-16 23:46:05.labeled
by None at 2022-11-16 23:45:24.labeled
by None at 2022-11-16 23:45:24.Superseded by #192.
closed
by None at 2022-11-16 23:45:26.head_ref_deleted
by None at 2022-11-16 23:45:27.referenced
by Kevin Reiss at 2022-11-16 22:01:50.merged
by Kevin Reiss at 2022-11-16 22:01:50.closed
by Kevin Reiss at 2022-11-16 22:01:50.head_ref_deleted
by Kevin Reiss at 2022-11-16 22:01:51.closed
by Kevin Reiss at 2022-11-17 00:57:00.wrong direction. Closing.
closed
by Chuck McCallum at 2022-11-16 21:28:05.head_ref_deleted
by Chuck McCallum at 2022-11-16 21:28:09.wrong direction. Closing.
closed
by Chuck McCallum at 2022-11-16 21:28:19.head_ref_deleted
by Chuck McCallum at 2022-11-16 21:28:22.wrong direction. Closing.
ready_for_review
by Chuck McCallum at 2022-11-16 19:17:49.closed
by Chuck McCallum at 2022-11-16 21:28:31.head_ref_deleted
by Chuck McCallum at 2022-11-16 21:28:35.wrong direction. Closing.
convert_to_draft
by Chuck McCallum at 2022-11-16 18:55:34.closed
by Chuck McCallum at 2022-11-16 21:28:44.head_ref_deleted
by Chuck McCallum at 2022-11-16 21:28:48.wrong direction. Closing.
closed
by Chuck McCallum at 2022-11-16 21:28:55.head_ref_deleted
by Chuck McCallum at 2022-11-16 21:28:59.@mccalluc Did you create this utilizing the rails-template? Or did you do a rails generate
I know @bess utilized the rails template for pdc-describe and thought it made things smoother.
Also, please note that I have added some basic branch protections for main
using https://github.com/pulibrary/tiger-data-app/settings/branch_protection_rules/31030809
wrong direction. Closing.
mentioned
by Bess Sadler at 2022-11-16 20:30:07.subscribed
by Bess Sadler at 2022-11-16 20:30:07.mentioned
by Chuck McCallum at 2022-11-16 20:30:07.subscribed
by Chuck McCallum at 2022-11-16 20:30:07.closed
by Chuck McCallum at 2022-11-16 21:29:51.head_ref_deleted
by Chuck McCallum at 2022-11-16 21:29:54.transferred
by Chuck McCallum at 2022-11-16 23:17:28.transferred
by Chuck McCallum at 2022-11-16 23:19:10.transferred
by Chuck McCallum at 2022-11-16 23:19:48.transferred
by Chuck McCallum at 2022-11-16 23:21:06.transferred
by Chuck McCallum at 2022-11-16 23:21:24.transferred
by Chuck McCallum at 2022-11-16 23:29:06.transferred
by Chuck McCallum at 2022-11-16 23:29:19.assigned
by None at 2022-11-16 12:49:38.referenced
by None at 2022-11-16 17:23:55.referenced
by None at 2022-11-16 17:24:56.referenced
by Hector Correa at 2022-11-16 20:15:23.referenced
by None at 2022-11-16 12:51:24.merged
by None at 2022-11-16 12:51:24.closed
by None at 2022-11-16 12:51:25.head_ref_deleted
by None at 2022-11-16 12:51:25.referenced
by None at 2022-11-16 12:55:49.merged
by None at 2022-11-16 12:55:49.closed
by None at 2022-11-16 12:55:49.head_ref_deleted
by None at 2022-11-16 12:55:50.closed
by None at 2022-11-16 12:55:50.closed
by Bess Sadler at 2022-11-16 15:29:26.referenced
by Hector Correa at 2022-11-16 20:15:22.merged
by Hector Correa at 2022-11-16 20:15:22.closed
by Hector Correa at 2022-11-16 20:15:22.head_ref_deleted
by Hector Correa at 2022-11-16 20:15:23.See code that @carolyncole implemented using multipart/form-data
encoding (instead of DIME) to upload a file. Her code is based on the Mediaflux Developer Guide (page 143): https://github.com/pulibrary/tiger-data-experiments/pull/45
I did not find in the Developer Guide a way to download the content of a file using multipart/form-data
encoding, but I am pretty sure there is one.
I will suggest we wait until we have a developer session with Arcitecta and ask about this.
We already have have a way to download content (using the DIME format) if we have to (see http_post
in https://github.com/pulibrary/tiger-data-experiments/blob/50dc63dbb0fe840cfbf2b030222cc8babae822de/ruby-hardcoded-api/mfclient.rb)
mentioned
by None at 2022-11-16 20:18:33.subscribed
by None at 2022-11-16 20:18:33.closed
by Hector Correa at 2022-11-16 21:07:33.Done
closed
by Chuck McCallum at 2022-11-16 23:24:47.We're not sure if we'll need this, or if we'll do it a different way.
Docs in the new repo are tracked by - https://github.com/pulibrary/tiger-data-app/issues/12
closed
by Chuck McCallum at 2022-11-16 23:28:28.not needed
closed
by Chuck McCallum at 2022-11-16 23:34:50.head_ref_deleted
by Chuck McCallum at 2022-11-16 23:35:04.referenced
by Chuck McCallum at 2022-11-16 23:38:29.merged
by Chuck McCallum at 2022-11-16 23:38:29.closed
by Chuck McCallum at 2022-11-16 23:38:29.head_ref_deleted
by Chuck McCallum at 2022-11-16 23:38:30.


{{pullRequest.repositoryName}}, from {{pullRequest.sourceRefName}} to {{pullRequest.targetRefName}}
by {{pullRequest.createdBy}}, at {{pullRequest.createdDate | date: dateFormat}}
Merge Conflicts
0">, {{reviewer.isRequired ? reviewer.displayName + '*' : reviewer.displayName}}


antibodies.Rmd
Since we acknowledge most experimental workflows need antibodies. We have implemented several functions to look for antibodies in vendor websites, as well as some helper functions to find the other posible aliases a gene might have.
Here is a simple example for a gene widely know to have an antibody available
## Loading required package: sctree
## Loading required package: Seurat
## Registered S3 method overwritten by 'R.oo':
## method from
## throw.default R.methodsS3
## Registered S3 method overwritten by 'GGally':
## method from
## +.gg ggplot2
##
## Attaching package: 'sctree'
## The following objects are masked from 'package:Seurat':
##
## FindAllMarkers, FindConservedMarkers, FindMarkers
## title vendor
## 1 Anti-CD11b antibody [EPR1344] Abcam
## 2 Anti-CD11b/ITGAM Antibody BosterBio
## 3 Anti-CD11b/ITGAM Picoband Antibody BosterBio
## 4 Anti-CD11b Rabbit Monoclonal Antibody BosterBio
## 5 Monoclonal Antibody to CD11b (human) MyBioSource.com
## 6 Anti-CD11b (Mouse) mAb MBL International
## specification
## 1 Applications: WB, IHC-p; Reactivity: Hu, Ms, Rt, Pg, RhMk; Conjugate/Tag: Unconjugated; Quantity: 10 µl, 40 µl, 100 µl
## 2 Applications: Western Blot (WB); Reactivity: Hu, Ms, Rt; Conjugate/Tag: Unconjugated; Quantity: 100ug/vial
## 3 Applications: WB, FCM, ICC, IHC-fr, IHC-p; Reactivity: Hu, Ms, Rt; Conjugate/Tag: Unconjugated; Quantity: 100ug/vial
## 4 Applications: WB, IF, IHC; Reactivity: Hu, Ms; Conjugate/Tag: Unconjugated; Quantity: 100ug/vial
## 5 Applications: Flow Cytometry (FCM); Reactivity: Human (Hu); Conjugate/Tag: Unconjugated; Quantity: 0.1 mg
## 6 Applications: Flow Cytometry (FCM); Reactivity: Mouse (Ms); Conjugate/Tag: Unconjugated; Quantity: 100 ug


Components¶
Workflow is comprised of a number of small, independent services that combine to create a distributed PaaS. All Workflow components are deployed as services (and associated controllers) in your Kubernetes cluster. If you are interested we have a more detailed exploration of the Workflow architecture.
All of the componentry for Workflow is built with composability in mind. If you need to customize one of the components for your specific deployment or need the functionality in your own project we invite you to give it a shot!
Controller¶
Project Location: deis/controller
The controller component is an HTTP API server which serves as the endpoint for
the deis
CLI. The controller provides all of the platform functionality as
well as interfacing with your Kubernetes cluster. The controller persists all
of its data to the database component.
Database¶
Project Location: deis/postgres
The database component is a managed instance of PostgreSQL which holds a majority of the platforms state. Backups and WAL files are pushed to object storage via WAL-E. When the database is restarted, backups are fetched and replayed from object storage so no data is lost.
Builder¶
Project Location: deis/builder
The builder component is responsible for accepting code pushes via Git and managing the build process of your Application. The builder process is:
- Receives incoming
git push
requests over SSH - Authenticates the user via SSH key fingerprint
- Authorizes the user's access to push code to the Application
- Starts the Application Build phase (see below)
- Triggers a new Release via the Controller
Builder currently supports both buildpack and Dockerfile based builds.
Project Location: deis/slugbuilder
For Buildpack-based deploys, the builder component will launch a one-shot Pod
in the deis
namespace. This pod runs slugbuilder
component which handles
default and custom buildpacks (specified by BUILDPACK_URL
). The "compiled"
application results in a slug, consisting of your application code and all of
its dependencies as determined by the buildpack. The slug is pushed to the
cluster-configured object storage for later execution. For more information
about buildpacks see using buildpacks.
Project Location: deis/dockerbuilder
For Applications which contain a Dockerfile
in the root of the repository,
builder
will instead launch the dockerbuilder
to package your application.
Instead of generating a slug, dockerbuilder
generates a Docker image (using
the underlying Docker engine). The completed image is pushed to the managed
Docker registry on cluster. For more information see using Dockerfiles.
Slugrunner¶
Project Location: deis/slugrunner
Slugrunner is the component responsible for executing buildpack-based Applications. Slugrunner receives slug information from the controller and downloads the application slug just before launching your application processes.
Object Storage¶
Project Location: deis/minio
All of the Workflow components that need to persist data will ship them to the object storage that was configured for the cluster.For example, database ships its WAL files, registry stores Docker images, and slugbuilder stores slugs.
Workflow supports either on or off-cluster storage. For production deployments we highly recommend that you configure off-cluster object storage.
To facilitate experimentation, development and test environments, the default charts for Workflow include on-cluster object storage via minio.
If you also feel comfortable using Kubernetes persistent volumes you may configure minio to use persistent storage available in your environment.
Registry¶
Project Location: deis/registry
The registry component is a managed docker registry which holds application images generated from the builder component. Registry persists the Docker image images to either local storage (in development mode) or to object storage configured for the cluster.
Router¶
Project Location: deis/router
The router component is based on Nginx and is responsible for routing
inbound HTTP(S) traffic to your applications. The default workflow charts
provision a Kubernetes service in the deis
namespace with a service type of
LoadBalancer
. Depending on your Kubernetes configuration, this may provision
a cloud-based loadbalancer automatically.
The router component uses Kubernetes annotations for both Application discovery as well as router configuration. For more detailed documentation and possible configuration view the router project documentation.
Logger: fluentd, logger¶
The logging subsystem consists of two components. Fluentd handles log shipping and logger maintains a ring-buffer of application logs.
Project Location: deis/fluentd
Fluentd is deployed to your Kubernetes cluster via Daemon Sets. Fluentd
subscribes to all container logs, decorates the output with Kubernetes metadata
and can be configured to drain logs to multiple destinations. By default,
fluentd ships logs to the logger component, which powers deis logs
.
Project Location: deis/logger
The logger
component receives log streams from fluentd
, collating by
Application name. Logger does not persist logs to disk, instead maintaining an
in-memory ring buffer. For more information on logger see the project
documentation.
Monitor¶
Project Location: deis/monitor
The monitoring subsystem consists of three components: Telegraf, InfluxDB and Grafana.
Telegraf is the is the metrics collection agent that runs using the daemon set API. It runs on every worker node in the cluster, fetches information about the pods currently running and ships it to InfluxDB.
InfluxDB is a database that stores the metrics collected by Telegraf. Out of the box, it does not persist to disk, but you can set it up to back it with a persisitent volume or swap this out with a more robust InfluxDB setup in a production setting.
Grafana is a standalone graphing application. It natively supports InfluxDB as a datasource and provides a robust engine for creating dashboards on top of timeseries data. Workflow provides a few dashboards out of the box for monitoring Deis Workflow and Kubernetes. The dashboards can be used as a starting point for creating more custom dashboards to suit a user's needs.
Workflow Manager¶
Project Location: deis/workflow-manager
Workflow Manager
will regularly check your cluster against the latest stable
components. If components are missing due to failure or are simply out of date,
Workflow operators will know at a glance. By default, this submits component
and version information to Deis' version service. If you prefer, you may
disable the function by setting WORKFLOW_MANAGER_CHECKVERSIONS
to false in
Workflow Manager's Deployment.


About Project:
"Streng Geheim" is a German phrase which means "Top Secret". This is an open-source project which provides Steganography functionality which means concealing a secret message, image, or file within another message, image, or file. This app uses images for that.
This app provides AES, RC5, RC6 and TripleDES data encryption and SHA-1, SHA-256 and MD-5 hashing algorithm for password protection. It uses XZ(LZMA2) data compression. You can choose any encryption and hashing algorithm to protect your data.
About Us:
Nothing much, just like to do cool stuff. I will appreciate if people contribute more to this project and help to make it better.
Project Link:
https://github.com/last-stand/StrengGeheim
More about Steganography:
Steganography
How it works?


The data we used in CSV format have two features weight and height .The Dependent value is Weight and the Independent value is Height.


Once you have completed your field data collection, you can use the Export button below to post your data back to the MapMint server.


Constructs a new Logger
.
A set of LoggerOptions
or name to use with the default options.
Throws SfdxError{ name: 'RedundantRootLogger' } More than one attempt is made to construct the root
Logger
.
The default LoggerLevel
when constructing new Logger
instances.
A list of all lower case LoggerLevel
names.
See LoggerLevel
The name of the root sfdx Logger
.
Add a field to all log lines for this logger. For convenience this
object is returned.
The name of the field to add.
The value of the field to be logged.
Adds a filter to be applied to all logged messages.
A function with signature (...args: any[]) => any[]
that transforms log message arguments.
Adds a file stream to this logger. Resolved or rejected upon completion of the addition.
The path to the log file. If it doesn't exist it will be created.
Adds a stream.
The stream configuration to add.
The default level of the stream.
Create a child logger, typically to add a few log record fields. For convenience this object is returned.
The name of the child logger that is emitted w/ log line as log:<name>
.
Additional fields included in all log lines for the child logger.
Close the logger, including any streams, and remove all listeners.
A function with signature (stream: LoggerStream) => void
to call for each stream with
the stream as an arg.
Logs at debug
level with filtering applied. For convenience this
object is returned.
Any number of arguments to be logged.
Logs at error
level with filtering applied. For convenience this
object is returned.
Any number of arguments to be logged.
Logs at fatal
level with filtering applied. For convenience this
object is returned.
Any number of arguments to be logged.
Gets an array of log line objects. Each element is an object that corresponds to a log line.
Gets the underlying Bunyan logger.
Gets the current level of this logger.
Gets the name of this logger.
Logs at info
level with filtering applied. For convenience this
object is returned.
Any number of arguments to be logged.
Reads a text blob of all the log lines contained in memory or the log file.
Set the logging level of all streams for this logger. If a specific level
is not provided, this method will
attempt to read it from the environment variable SFDX_LOG_LEVEL
, and if not found,
{@link Logger.DEFAULT_LOG_LEVEL} will be used instead. For convenience this
object is returned.
Compares the requested log level with the current log level. Returns true if the requested log level is greater than or equal to the current log level.
The requested log level to compare against the currently set log level.
Logs at trace
level with filtering applied. For convenience this
object is returned.
Any number of arguments to be logged.
Use in-memory logging for this logger instance instead of any parent streams. Useful for testing. For convenience this object is returned.
WARNING: This cannot be undone for this logger instance.
Logs at warn
level with filtering applied. For convenience this
object is returned.
Any number of arguments to be logged.
Gets a numeric LoggerLevel
value by string name.
The level name to convert to a LoggerLevel
enum value.
Throws SfdxError{ name: 'UnrecognizedLoggerLevelName' } The level name was not case-insensitively recognized as a valid LoggerLevel
value.
Gets the root logger with the default level and file stream.
A logging abstraction powered by Bunyan that provides both a default logger configuration that will log to
sfdx.log
, and a way to create custom loggers based on the same foundation.// Gets the root sfdx logger const logger = await Logger.root(); // Creates a child logger of the root sfdx logger with custom fields applied const childLogger = await Logger.child('myRootChild', {tag: 'value'}); // Creates a custom logger unaffiliated with the root logger const myCustomLogger = new Logger('myCustomLogger'); // Creates a child of a custom logger unaffiliated with the root logger with custom fields applied const myCustomChildLogger = myCustomLogger.child('myCustomChild', {tag: 'value'});
See https://github.com/cwallsfdc/node-bunyan
See https://developer.salesforce.com/docs/atlas.en-us.sfdx_dev.meta/sfdx_dev/sfdx_dev_cli_log_messages.htm


--- layout: post title: 'World Telemedia Day 2: Making it work across devices, Hila Dahan of Adamind' date: 2005-11-17 00:00:00 categories: digital-media mobile software-development ---
World Telemedia Day 2: Making it work across devices, Hila Dahan of Adamind
P2P and A2P multimedia services are growing. (What's A2P?)
Many devices. Many content types. Many data formats.
Transcoding as an alternative to manually versioning.
Converged services will be realised through transcoding.  They're building their own database of device attributes, optimisation rules, etc.


This application gives the daily tally of confirmed cases across the globe.
Total cases reported as of today
Total new cases reported since last day


By default, the specified command will run in the directory containing this
Webmin module's programs, which will be something like
/usr/libexec/webmin/custom. This option can be used to specify a
different directory, which can be useful for avoiding the need to use full
paths for the command and its parameters.


Extracting emotions from face images using Affectiva API
Use this script to extract facial expressions from a face image file using
the Affectiva javascript API.
If you would like to run this program locally or need to process videos,
you can check out the github code at https://github.com/cosanlab/affectiva-api-app
and the tutorial here.
When finished processing, your browser
will download the data file automatically.
An upload button will appear here after the detector has been intialized.
You can upload an image or multiple images to process at the same time.
Results will be automatically downloaded as output.json.


Omics Pipe Tutorial – Creating a Custom Pipeline Script¶
A pipeline script is a .py file that has the steps in the pipeline that you want to run in your analysis. To create a custom pipeline, you will create a new Python script (*.py) file and place it in your working directory (or wherever you want). The available analysis steps built-in to omics_pipe are available in the (List of currently available omics_pipe analysis steps).
You may add new modules directly to the module directory (see Adding Custom Modules), and they will become available steps that you can use in your custom pipeline.
- There are three steps to creating a custom pipeline:
- Designing the structure of your pipeline
- Creating the script
- Updating your parameters file
The section below details each of these steps.
Designing the structure of the pipeline¶
Omics_pipe depends upon the pipelining module Ruffus to handle the automation. Please read the documentation at the Ruffus website http://www.ruffus.org.uk/ for more information. To design your pipeline, you need to decide
- What analysis modules you want to run,
- What order you want the analysis modules to run in,
- Which, if any, of the analysis modules depend upon the results from another analysis module.
For example, we will create a custom pipeline that runs fastqc, star and htseq (depends on output from star).
Creating the script¶
To create the script, create a text file name custom_script.py (or whatever name you choose). At the top of the file, cut and copy this text:
#!/usr/bin/env python
from ruffus import *
import sys
import os
import time
import datetime
import drmaa
from omics_pipe.utils import *
from omics_pipe.parameters.default_parameters import default_parameters
p = Bunch(default_parameters)
os.chdir(p.WORKING_DIR)
now = datetime.datetime.now()
date = now.strftime("%Y-%m-%d %H:%M")
print p
for step in p.STEPS:
vars()['inputList_' + step] = []
for sample in p.SAMPLE_LIST:
vars()['inputList_' + step].append([sample, "%s/%s_%s_completed.flag" % (p.FLAG_PATH, step, sample)])
print vars()['inputList_' + step]
After this has been completed, you will need to import each of the analysis modules that you will use in your pipeline. For each analysis module, write the line below (fill in analysis_name with the name of the analysis module):
from omics_pipe.modules.analysis_name import analysis_name
In our example, you will have three lines (see below):
from omics_pipe.modules.fastqc import fastqc
from omics_pipe.modules.star import star
from omics_pipe.modules.htseq import htseq
Now you are ready to write the functions to run each of these steps in the analysis. For each step in our analysis pipeline, we will need to write a function. You can cut and copy these from the pre-packaged analysis pipeline scripts (or eventually a function reference) or create them. Each function needs to have two decorators from Ruffus: @parallel(inputList_analysis_name) to specify that the pipeline should run in parallel for more than one sample and @check_if_uptodate(check_file_exists) to call a function to check if that step in the pipeline is up to date. Name each function with the name of the analysis prefixed by “run_.” The function input should always be (sample, analysis_name_flag). Within the function, you will call the analysis module that you loaded above. If you want an analysis module to run only after a module it depends upon finishes, you must add the @follows() Ruffus decorator before the function, with the name of the step that it depends upon. For example, if htseq needs to run after star, you would put @follows(run_star) above the run_htseq function. If you have steps that do not have functions that are dependent upon them, you can create a more complex pipeline structure by creating a “Last Function” that ties together all steps of your pipeline. The last function below is an example of such a function, and it also produces a PDF diagram of your pipeline when it completes. The functions for our example are below.
@parallel(inputList_fastqc)
@check_if_uptodate(check_file_exists)
def run_fastqc(sample, fastqc_flag):
fastqc(sample, fastqc_flag)
return
@parallel(inputList_star)
@check_if_uptodate(check_file_exists)
def run_star(sample, star_flag):
star(sample, star_flag)
return
@parallel(inputList_htseq)
@check_if_uptodate(check_file_exists)
@follows(run_star)
def run_htseq(sample, htseq_flag):
htseq(sample, htseq_flag)
return
@parallel(inputList_last_function)
@check_if_uptodate(check_file_exists)
@follows(run_fastqc, run_htseq)
def last_function(sample, last_function_flag):
print "PIPELINE HAS FINISHED SUCCESSFULLY!!! YAY!"
pipeline_graph_output = p.FLAG_PATH + "/pipeline_" + sample + "_" + str(date) + ".pdf"
pipeline_printout_graph (pipeline_graph_output,'pdf', step, no_key_legend=False)
stage = "last_function"
flag_file = "%s/%s_%s_completed.flag" % (p.FLAG_PATH, stage, sample)
open(flag_file, 'w').close()
return
Once you have created all of the functions for each step of your pipeline, cut and copy the code below to the bottom of your script:
if __name__ == '__main__':
pipeline_run(p.STEP, multiprocess = p.PIPE_MULTIPROCESS, verbose = p.PIPE_VERBOSE, gnu_make_maximal_rebuild_mode = p.PIPE_REBUILD)
At this point, please save your script and move on to step 3.
Updating your parameters file¶
In order for your script to run successfully, you need to configure your parameter file so that each analysis module has the necessary parameters to execute successfully. The full list of parameters for all modules in the current version of omics_pipe are located in the omics_pipe/parameters/default_parameters.py file (and eventually organized somewhere). You can view the list of necessary parameters for each analysis module by importing the analysis module into an interactive python session (from omics_pipe.modules.analysis_module import analysis_module) and typing analysis_module.__doc__. The parameters necessary for that analysis module will be listed under “parameters from parameters file.” These parameters must be put into your parameters.yaml file and spelled exactly as shown (including all caps). Below is the list of parameters that are necessary to run omics_pipe in addition to the module specific parameters.
SAMPLE_LIST: [test, test1]
STEP: run_last_function
STEPS: [fastqc, star, htseq, last_function]
RAW_DATA_DIR: /gpfs/group/sanford/patient/SSKT/test_patient/RNA/RNA_seq/data
FLAG_PATH: /gpfs/group/sanford/patient/SSKT/test_patient/RNA/RNA_seq/logs/flags
LOG_PATH: /gpfs/group/sanford/patient/SSKT/test_patient/RNA/RNA_seq/logs
WORKING_DIR: /gpfs/home/kfisch/virt_env/virt2/lib/python2.6/site-packages/omics_pipe-1.0.7-py2.6.egg/omics_pipe/scripts
ENDS: PE
PIPE_MULTIPROCESS: 100
PIPE_REBUILD: 'True'
PIPE_VERBOSE: 5
RESULTS_EMAIL<EMAIL_ADDRESS>TEMP_DIR: /scratch/kfisch
DPS_VERSION: '1.3.1111'
QUEUE: bigmem
PARAMS_FILE: /gpfs/home/kfisch/omics_pipe_docs/test_params.yaml
USERNAME: kfisch
GENOME: /gpfs/group/databases/Homo_sapiens/UCSC/hg19/Sequence/WholeGenomeFasta/genome.fa
CHROM: /gpfs/group/databases/Homo_sapiens/UCSC/hg19/Sequence/Chromosomes
REF_GENES: /gpfs/group/databases/Homo_sapiens/UCSC/hg19/Annotation/Genes/genes.gtf
STAR_INDEX: /gpfs/group/databases/Homo_sapiens/UCSC/hg19/star_genome
BOWTIE_INDEX: /gpfs/group/databases/Homo_sapiens/UCSC/hg19/Sequence/Bowtie2Index/genome
Once you have all of the necessary parameters in your parameters.yaml file, for your custom script you will need to change the STEP and STEPS parameters. In the STEP parameter, you will write the name of the last function in your pipeline that you want to run, which should be configured so that it captures all steps in the pipeline (as in the example above). Make sure to put run_ in front of this, since you are calling the function, not the analysis module. In order for omics_pipe to know what steps you have in your pipeline, you need to list each analysis module name in the STEPS parameter separated with commas (without run_ in the prefix).
Running omics_pipe with a custom pipeline script When you call the omics_pipe function, you will specify the path to your custom script using the command
omics_pipe custom --custom_script_path ~/path/to/the/script --custom_script_name name_of_customscript /path/to/parameters.yaml
This will automatically load your custom script and run through the steps in your pipeline using the default modules available in omics_pipe.


Main
Explorer
Edge
Settings
Spotify
Terminal
General
Edge
Settings
File Explorer
Edge Browser
Settings
System
Devices
Internet
Personalization
Accounts
Time
Update and Security
Designed by
Tanner Villarete
Preview
Aa
Background
Choose your picture
Browse
Accent color
Start Menu
No Lock Screen settings yet
No Theme settings yet
No Start Menu settings yet
No Taskbar settings yet
lol
Update and Security
Power
Custom Wallpaper
Spotify
Command Prompt


Total Management is a free, tableless, W3C-compilant web design layout by template
World.
This template has been tested and proven compatible with all major browser
enviorments and operating systems.
You are free to modify the design to suit your
tastes in any way you like.
We only ask you to not remove "Design by Template World" and the link
http://www.templateworld.com from
the footer of the template.
If you are interested in seeing more of our free web temlate designs
feel free to visit our
website, Template World.
We intend to add at least 25 new free templates in the coming month.


NetCDF: Network Common Data Form
Driver short name
netCDF
Build dependencies
libnetcdf
This format is supported for read and write access. This page only describes the raster support (you can find documentation for the vector side) NetCDF is an interface for array-oriented data access and is used for representing scientific data.
The fill value metadata or missing_value backward compatibility is preserved as NODATA value when available.
NOTE: Implemented as netcdfdataset.cpp.
Driver capabilities
Supports CreateCopy()
This driver supports the GDALDriver::CreateCopy()
operation
Supports Create()
This driver supports the GDALDriver::Create()
operation
Supports Georeferencing
This driver supports georeferencing
Multiple Image Handling (Subdatasets)
Network Command Data Form is a container for several different arrays most used for storing scientific dataset. One NetCDF file may contain several datasets. They may differ in size, number of dimensions and may represent data for different regions.
If the file contains only one NetCDF array which appears to be an image, it may be accessed directly, but if the file contains multiple images it may be necessary to import the file via a two step process.
The first step is to get a report of the components images (dataset) in the file using gdalinfo, and then to import the desired images using gdal_translate. The gdalinfo utility lists all multidimensional subdatasets from the input NetCDF file.
The name of individual images are assigned to the SUBDATASET_n_NAME metadata item. The description for each image is found in the SUBDATASET_n_DESC metadata item. For NetCDF images will follow this format: NETCDF:filename:variable_name
where filename is the name of the input file, and variable_name is the dataset selected within the file.
On the second step you provide this name for gdalinfo to get information about the dataset or gdal_translate to read dataset.
For example, we want to read data from a NetCDF file:
$ gdalinfo sst.nc
Driver: netCDF/Network Common Data Format
Size is 512, 512
Coordinate System is `'
Metadata:
NC_GLOBAL#title=IPSL model output prepared for IPCC Fourth Assessment SRES A2 experiment
NC_GLOBAL#institution=IPSL (Institut Pierre Simon Laplace, Paris, France)
NC_GLOBAL#source=IPSL-CM4_v1 (2003) : atmosphere : LMDZ (IPSL-CM4_IPCC, 96x71x19) ; ocean ORCA2 (ipsl_cm4_v1_8, 2x2L31); sea ice LIM (ipsl_cm4_v
NC_GLOBAL#contact=Sebastien Denvil<EMAIL_ADDRESS>NC_GLOBAL#project_id=IPCC Fourth Assessment
NC_GLOBAL#table_id=Table O1 (13 November 2004)
NC_GLOBAL#experiment_id=SRES A2 experiment
NC_GLOBAL#realization=1
NC_GLOBAL#cmor_version=9.600000e-01
NC_GLOBAL#Conventions=CF-1.0
NC_GLOBAL#history=YYYY/MM/JJ: data generated; YYYY/MM/JJ+1 data transformed At 16:37:23 on 01/11/2005, CMOR rewrote data to comply with CF standards and IPCC Fourth Assessment requirements
NC_GLOBAL#references=Dufresne et al, Journal of Climate, 2015, vol XX, p 136
NC_GLOBAL#comment=Test drive
Subdatasets:
SUBDATASET_1_NAME=NETCDF:"sst.nc":lon_bnds
SUBDATASET_1_DESC=[180x2] lon_bnds (64-bit floating-point)
SUBDATASET_2_NAME=NETCDF:"sst.nc":lat_bnds
SUBDATASET_2_DESC=[170x2] lat_bnds (64-bit floating-point)
SUBDATASET_3_NAME=NETCDF:"sst.nc":time_bnds
SUBDATASET_3_DESC=[24x2] time_bnds (64-bit floating-point)
SUBDATASET_4_NAME=NETCDF:"sst.nc":tos
SUBDATASET_4_DESC=[24x170x180] sea_surface_temperature (32-bit floating-point)Corner Coordinates:
Upper Left ( 0.0, 0.0)
Lower Left ( 0.0, 512.0)
Upper Right ( 512.0, 0.0)
Lower Right ( 512.0, 512.0)
Center ( 256.0, 256.0)
This NetCDF files contain 4 datasets, lon_bnds, lat_bnds, tim_bnds and
tos. Now select the subdataset, described as: NETCDF:"sst.nc":tos
[24x17x180] sea_surface_temperature (32-bit floating-point)
and get the information about the number of bands there is inside this
variable.
$ gdalinfo NETCDF:"sst.nc":tos
Driver: netCDF/Network Common Data Format
Size is 180, 170
Coordinate System is `'
Origin = (1.000000,-79.500000)
Pixel Size = (1.98888889,0.99411765)
Metadata:
NC_GLOBAL#title=IPSL model output prepared for IPCC Fourth Assessment SRES A2 experiment
NC_GLOBAL#institution=IPSL (Institut Pierre Simon Laplace, Paris, France)
.... More metadata
time#standard_name=time
time#long_name=time
time#units=days since 2001-1-1
time#axis=T
time#calendar=360_day
time#bounds=time_bnds
time#original_units=seconds since 2001-1-1
Corner Coordinates:
Upper Left ( 1.0000000, -79.5000000)
Lower Left ( 1.0000000, 89.5000000)
Upper Right ( 359.000, -79.500)
Lower Right ( 359.000, 89.500)
Center (<PHONE_NUMBER>, 5.0000000)
Band 1 Block=180x1 Type=Float32, ColorInterp=Undefined
NoData Value=1e+20
Metadata:
NETCDF_VARNAME=tos
NETCDF_DIMENSION_time=15
NETCDF_time_units=days since 2001-1-1
Band 2 Block=180x1 Type=Float32, ColorInterp=Undefined
NoData Value=1e+20
Metadata:
NETCDF_VARNAME=tos
NETCDF_DIMENSION_time=45
NETCDF_time_units=days since 2001-1-1
.... More Bands
Band 22 Block=180x1 Type=Float32, ColorInterp=Undefined
NoData Value=1e+20
Metadata:
NETCDF_VARNAME=tos
NETCDF_DIMENSION_time=645
NETCDF_time_units=days since 2001-1-1
Band 23 Block=180x1 Type=Float32, ColorInterp=Undefined
NoData Value=1e+20
Metadata:
NETCDF_VARNAME=tos
NETCDF_DIMENSION_time=675
NETCDF_time_units=days since 2001-1-1
Band 24 Block=180x1 Type=Float32, ColorInterp=Undefined
NoData Value=1e+20
Metadata:
NETCDF_VARNAME=tos
NETCDF_DIMENSION_time=705
NETCDF_time_units=days since 2001-1-1
gdalinfo displays the number of bands into this subdataset. There are metadata attached to each band. In this example, the metadata informs us that each band correspond to an array of monthly sea surface temperature from January 2001. There are 24 months of data in this subdataset. You may also use gdal_translate for reading the subdataset.
Note that you should provide exactly the contents of the line marked SUBDATASET_n_NAME to GDAL, including the NETCDF: prefix.
The NETCDF prefix must be first. It triggers the subdataset NetCDF driver. This driver is intended only for importing remote sensing and geospatial datasets in form of raster images. If you want explore all data contained in NetCDF file you should use another tools.
Starting with GDAL 3.5, the VARIABLES_AS_BANDS=YES
open option can be
used to indicate to the driver that if the netCDF file only contains
2D variables of the same type and indexed by the same dimensions, then they
should be reported as multiple bands of a same dataset.
$ gdalinfo autotest/gdrivers/data/netcdf/two_vars_scale_offset.nc -oo VARIABLES_AS_BANDS=YES
Driver: netCDF/Network Common Data Format
Files: autotest/gdrivers/data/netcdf/two_vars_scale_offset.nc
Size is 21, 21
Metadata:
NC_GLOBAL#Conventions=COARDS/CF-1.0
x#actual_range={-10,10}
x#long_name=x
y#actual_range={-10,10}
y#long_name=y
z#add_offset=1.5
z#long_name=z
z#scale_factor=0.01
Corner Coordinates:
Upper Left ( 0.0, 0.0)
Lower Left ( 0.0, 21.0)
Upper Right ( 21.0, 0.0)
Lower Right ( 21.0, 21.0)
Center ( 10.5, 10.5)
Band 1 Block=21x1 Type=Float32, ColorInterp=Undefined
NoData Value=9.96920996838686905e+36
Offset: 1.5, Scale:0.01
Metadata:
add_offset=1.5
long_name=z
NETCDF_VARNAME=z
scale_factor=0.01
Band 2 Block=21x1 Type=Float32, ColorInterp=Undefined
NoData Value=9.96920996838686905e+36
Offset: 2.5, Scale:0.1
Metadata:
add_offset=2.5
long_name=q
NETCDF_VARNAME=q
scale_factor=0.1
Dimension
The NetCDF driver assume that data follows the CF-1 convention from UNIDATA The dimensions inside the NetCDF file use the following rules: (Z,Y,X). If there are more than 3 dimensions, the driver will merge them into bands. For example if you have an 4 dimension arrays of the type (P, T, Y, X). The driver will multiply the last 2 dimensions (P*T). The driver will display the bands in the following order. It will first increment T and then P. Metadata will be displayed on each band with its corresponding T and P values.
Georeference
There is no universal way of storing georeferencing in NetCDF files. The driver first tries to follow the CF-1 Convention from UNIDATA looking for the Metadata named "grid_mapping". If "grid_mapping" is not present, the driver will try to find an lat/lon grid array to set geotransform array. The NetCDF driver verifies that the Lat/Lon array is equally spaced.
New in version 3.4: crs_wkt attribute support
If those 2 methods fail, NetCDF driver will try to read the following metadata directly and set up georeferencing.
spatial_ref (Well Known Text)
GeoTransform (GeoTransform array)
or,
Northernmost_Northing
Southernmost_Northing
Easternmost_Easting
Westernmost_Easting
See also the configuration options GDAL_NETCDF_VERIFY_DIMS and GDAL_NETCDF_IGNORE_XY_AXIS_NAME_CHECKS which control this behavior.
Open options
The following open options are available:
HONOUR_VALID_RANGE=[YES/NO]: Defaults to
YES
. Whether to set to nodata pixel values outside of the validity range indicated by valid_min, valid_max or valid_range attributes.IGNORE_XY_AXIS_NAME_CHECKS=[YES/NOA]: (GDAL >= 3.4.2) Defaults to
NO
. Whether X/Y dimensions should be always considered as geospatial axis, even if the lack conventional attributes confirming it.VARIABLES_AS_BANDS=[YES/NO]: (GDAL >= 3.5) Defaults to
NO
. If set to YES, and if the netCDF file only contains 2D variables of the same type and indexed by the same dimensions, then they should be reported as multiple bands of a same dataset. Default is NO (that is each variable will be reported as a separate subdataset)ASSUME_LONGLAT=[YES/NO]: (GDAL >= 3.7) Defaults to
NO
. Whether a Geographic CRS should be assumed and applied when, none has otherwise been found, a meaningful geotransform has been found, and that geotransform is within the bounds -180,360 -90,90, if YES assume OGC:CRS84.
Creation Issues
This driver supports creation of NetCDF file following the CF-1 convention. You may create set of 2D datasets. Each variable array is named Band1, Band2, ... BandN.
Each band will have metadata tied to it giving a short description of the data it contains.
GDAL NetCDF Metadata
All NetCDF attributes are transparently translated as GDAL metadata.
The translation follow these directives:
Global NetCDF metadata have a NC_GLOBAL tag prefixed.
Dataset metadata have their variable name prefixed.
Each prefix is followed by a # sign.
The NetCDF attribute follows the form: name=value.
Example:
$ gdalinfo NETCDF:"sst.nc":tos
Driver: netCDF/Network Common Data Format
Size is 180, 170
Coordinate System is `'
Origin = (1.000000,-79.500000)
Pixel Size = (1.98888889,0.99411765)
Metadata:
NetCDF global attributes
NC_GLOBAL#title=IPSL model output prepared for IPCC Fourth Assessment SRES A2 experiment
Variables attributes for: tos, lon, lat and time
tos#standard_name=sea_surface_temperature
tos#long_name=Sea Surface Temperature
tos#units=K
tos#cell_methods=time: mean (interval: 30 minutes)
tos#_FillValue=1.000000e+20
tos#missing_value=1.000000e+20
tos#original_name=sosstsst
tos#original_units=degC
tos#history= At 16:37:23 on 01/11/2005: CMOR altered the data in the following ways: added 2.73150E+02 to yield output units; Cyclical dimension was output starting at a different lon;
lon#standard_name=longitude
lon#long_name=longitude
lon#units=degrees_east
lon#axis=X
lon#bounds=lon_bnds
lon#original_units=degrees_east
lat#standard_name=latitude
lat#long_name=latitude
lat#units=degrees_north
lat#axis=Y
lat#bounds=lat_bnds
lat#original_units=degrees_north
time#standard_name=time
time#long_name=time
time#units=days since 2001-1-1
time#axis=T
time#calendar=360_day
time#bounds=time_bnds
time#original_units=seconds since 2001-1-1
On writing, when using the CreateCopy() interface or gdal_translate, dataset level metadata that follows the naming convention NC_GLOBAL#key=value will be used to write the netCDF attributes. Metadata set at the band level using key=value will also be used to write variable attributes.
Product specific behavior
Sentinel 5
New in version 3.4.
The most verbose metadata is reported in the json:ISO_METADATA
,
json:ESA_METADATA
, json:EOP_METADATA
, json:QA_STATISTICS
,
json:GRANULE_DESCRIPTION
, json:ALGORITHM_SETTINGS
and json:SUPPORT_DATA
metadata domains.
Can be discovered for example with:
gdalinfo -mdd all -json S5P.nc
Creation Options
FORMAT=[NC/NC2/NC4/NC4C]: Defaults to
NC
. Set the NetCDF file format to use. NC2 is normally supported by recent NetCDF installations, but NC4 and NC4C are available if NetCDF was compiled with NetCDF-4 (and HDF5) support.COMPRESS=[NONE/DEFLATE]: Set the compression to use. DEFLATE is only available if NetCDF has been compiled with NetCDF-4 support. NC4C format is the default if DEFLATE compression is used.
ZLEVEL=1-9: Defaults to
1
. Set the level of compression when using DEFLATE compression. A value of 9 is best, and 1 is least compression. The default is 1, which offers the best time/compression ratio.WRITE_BOTTOMUP=[YES/NO]: Defaults to
YES
. Set the y-axis order for export, overriding the order detected by the driver. NetCDF files are usually assumed "bottom-up", contrary to GDAL's model which is "north up". This normally does not create a problem in the y-axis order, unless there is no y axis geo-referencing. The default for this setting is YES, so files will be exported in the NetCDF default "bottom-up" order. For import seeGDAL_NETCDF_BOTTOMUP
below.WRITE_GDAL_TAGS=[YES/NO]: Define if GDAL tags used for georeferencing (spatial_ref and GeoTransform) should be exported, in addition to CF tags. Not all information is stored in the CF tags (such as named datums and EPSG codes), therefore the driver exports these variables by default. In import the CF "grid_mapping" variable takes precedence and the GDAL tags are used if they do not conflict with CF metadata. In GDAL 4, spatial_ref will not be exported. The crs_wkt CF metatata attribute will be used instead.
WRITE_LONLAT=[YES/NO/IF_NEEDED]: Define if CF lon/lat variables are written to file. Default is YES for geographic SRS and NO for projected SRS. This is normally not necessary for projected SRS as GDAL and many applications use the X/Y dimension variables and CF projection information. Use of IF_NEEDED option creates lon/lat variables if the projection is not part of the CF-1.5 standard.
TYPE_LONLAT=[float/double]: Set the variable type to use for lon/lat variables. Default is double for geographic SRS and float for projected SRS. If lon/lat variables are written for a projected SRS, the file is considerably large (each variable uses X*Y space), therefore
TYPE_LONLAT=float
andCOMPRESS=DEFLATE
are advisable in order to save space.PIXELTYPE=[DEFAULT/SIGNEDBYTE]: By setting this to SIGNEDBYTE, a new Byte file can be forced to be written as signed byte. Starting with GDAL 3.7, this option is deprecated and Int8 should rather be used.
WRITE_GDAL_VERSION=[YES/NO]: (GDAL >= 3.5.0) Defaults to
YES
. Define if a "GDAL" text global attribute should be added on file creation with the GDAL versionWRITE_GDAL_HISTORY=[YES/NO]: (GDAL >= 3.5.0) Defaults to
YES
. Define if the "history" global attribute should be prepended with a date/time and GDAL information.
Creation of multidimensional files with CreateCopy() 2D raster API
Starting with GDAL 3.1, the preferred way of creating > 2D files is to use the the Multidimensional Raster Data Model API. However it is possible to create such files with the 2D raster API using the CreateCopy() method (note that at time of writing, this is not supported using the Create() method).
The NETCDF_DIM_EXTRA={dim1_name,...dimN_name}
metadata item must be set on the
source dataset, where dim1_name is the name of the slowest varying dimension
and dimN_name the name of the fastest varying one.
For each extra dimension, the NETCDF_DIM_{dim_name}_DEF={dimension_size,netcdf_data_type}
metadata item must be set where dimension_size is the size of the dimension
(number of samples along that dimension) and netcdf_data_type is the integer
value for the netCDF data type of the corresponding indexing variable. Among the most useful
data types:
4 for Int
5 for Float
6 for Double
10 for Int64
The NETCDF_DIM_{dim_name}_VALUES={value1,...valueN}
is set to define the
values of the indexing variable corresponding to dimension.
dim_name#attribute
metadata items can also be set to define the attributes
of the indexing variable of the dimension.
Example of creation of a Time,Z,Y,X 4D file in Python:
# Create in-memory file with required metadata to define the extra >2D
# dimensions
size_z = 2
size_time = 3
src_ds = gdal.GetDriverByName('MEM').Create('', 4, 3, size_z * size_time)
src_ds.SetMetadataItem('NETCDF_DIM_EXTRA', '{time,Z}')
# 6 is NC_DOUBLE
src_ds.SetMetadataItem('NETCDF_DIM_Z_DEF', f"{{{size_z},6}}")
src_ds.SetMetadataItem('NETCDF_DIM_Z_VALUES', '{1.25,2.50}')
src_ds.SetMetadataItem('Z#axis', 'Z')
src_ds.SetMetadataItem('NETCDF_DIM_time_DEF', f"{{{size_time},6}}")
src_ds.SetMetadataItem('NETCDF_DIM_time_VALUES', '{1,2,3}')
src_ds.SetMetadataItem('time#axis', 'T')
src_ds.SetGeoTransform([2,1,0,49,0,-1])
# Create netCDF file
gdal.GetDriverByName('netCDF').CreateCopy('out.nc', src_ds)
Configuration Options
GDAL_NETCDF_BOTTOMUP=[YES/NO]: Set the y-axis order for import, overriding the order detected by the driver. This option is usually not needed unless a specific dataset is causing problems (which should be reported on GitHub).
GDAL_NETCDF_VERIFY_DIMS=[YES/STRICT]: Defaults to
YES
. Try to guess which dimensions represent the latitude and longitude only by their attributes (STRICT) or also by guessing the name (YES).GDAL_NETCDF_IGNORE_XY_AXIS_NAME_CHECKS=[YES/NO]: Defaults to
NO
. Whether X/Y dimensions should be always considered as geospatial axis, even if the lack conventional attributes confirming it.GDAL_NETCDF_ASSUME_LONGLAT=[YES/NO]: (GDAL >= 3.7) Defaults to
NO
. Whether a Geographic CRS should be assumed and applied when, none has otherwise been found, a meaningful geotransform has been found, and that geotransform is within the bounds -180,360 -90,90, if YES assume OGC:CRS84.
VSI Virtual File System API support
Since GDAL 2.4, and with Linux kernel >=4.3 and libnetcdf >=4.5, read operations on /vsi file systems are supported.
NetCDF-4 groups support on reading (GDAL >= 3.0)
The driver has undergone significant changes in GDAL 3.0 to support NetCDF-4 groups on reading:
Explore recursively all nested groups to create the subdatasets list
Subdatasets in nested groups use the /group1/group2/.../groupn/var standard NetCDF-4 convention, except for variables in the root group which do not have a leading slash for backward compatibility with the NetCDF-3 driver
Global attributes of each nested group are also collected in the GDAL dataset metadata, using the same convention /group1/group2/.../groupn/NC_GLOBAL#attr_name, except for the root group which do not have a leading slash for backward compatibility
When searching for a variable containing auxiliary information on the selected subdataset, like coordinate variables or grid_mapping, we now also search in parent groups and their children as specified in Support of groups in CF
Multidimensional API support
New in version 3.1.
The netCDF driver supports the Multidimensional Raster Data Model for reading and creation operations.
The GDALGroup::GetGroupNames()
method supports the following options:
GROUP_BY=SAME_DIMENSION. If set, single-dimensional variables will be exposed as a "virtual" subgroup. This enables the user to get a clearer organization of variables, for example in datasets where variables belonging to different trajectories are indexed by different dimensions but mixed in the same netCDF group.
The GDALGroup::OpenGroup()
method supports the following options:
GROUP_BY=SAME_DIMENSION. See above description
The GDALGroup::GetMDArrayNames()
method supports the following options:
SHOW_ALL=YES/NO. Defaults to NO. If set to YES, all variables will be listed.
SHOW_ZERO_DIM=YES/NO. Defaults to NO. If set to NO, variables with 0-dimension will not be listed.
SHOW_COORDINATES=YES/NO. Defaults to YES. If set to NO, variables refererenced in the
coordinates
attribute of another variable will not be listed.SHOW_BOUNDS=YES/NO. Defaults to YES. If set to NO, variables refererenced in the
bounds
attribute of another variable will not be listed.SHOW_INDEXING=YES/NO. Defaults to YES. If set to NO, single-dimensional variables whose name is equal to the name of their indexing variable will not be listed.
SHOW_TIME=YES/NO. Defaults to YES. If set to NO, single-dimensional variables whose
standard_name
attribute is "time" will not be listed.GROUP_BY=SAME_DIMENSION. If set, single-dimensional variables will not be listed
The GDALGroup::OpenMDArray()
method supports the following options:
USE_DEFAULT_FILL_AS_NODATA=YES/NO. (GDAL >= 3.6) Defaults to NO. If set to YES, the default fill value will be used as nodata when there is no _FillValue or missing_value attribute (except on variables of type Byte, UByte, Char)
The GDALGroup::CreateMDArray()
method supports the following options:
NC_TYPE=NC_CHAR/NC_BYTE/NC_INT64/NC_UINT64: to overload the netCDF data type normally deduced from the GDAL data type passed to CreateMDArray(). NC_CHAR can only be used for strings of a fixed size.
BLOCKSIZE=size_dim0,size_dim1,...,size_dimN: to set the netCDF chunk size, as set by nc_def_var_chunking(). There must be exactly as many values as the number of dimensions passed to CreateMDArray()
COMPRESS=DEFLATE: to ask for deflate compression
ZLEVEL=number: DEFLATE compression level (1-9)
CHECKSUM=YES/NO: Whether to turn on Fletcher32 checksums. Checksum generation requires chunking, and if no explicit chunking has been asked with the BLOCKSIZE option, a default one will be used. Defaults to NO.
FILTER=filterid,param1,...,paramN: Define a filter (typically a compression method) used for writing. This should be a list of numeric values, separated by commas. The first value is the filter id (list of potential values at https://support.hdfgroup.org/services/contributions.html#filters) and following values are per-filter parameters. More details about netCDF-4 filter support at https://www.unidata.ucar.edu/software/netcdf/docs/md__Users_wfisher_Desktop_docs_netcdf-c_docs_filters.html
Driver building
This driver is compiled with the UNIDATA NetCDF library.
You need to download or compile the NetCDF library before configuring GDAL with NetCDF support.
See NetCDF GDAL wiki for build instructions and information regarding HDF4, NetCDF-4 and HDF5.


@remarks The UnicodeEncoding object that is returned by this property may not have the appropriate behavior for your application. It uses replacement fallback to replace each string that it cannot encode and each byte that it cannot decode with a question mark ("?") character. Instead, you can call the NewUnicodeEncoding(Boolean, Boolean, Boolean) constructor to instantiate a big endian UnicodeEncoding object whose fallback is either an EncoderFallbackException or a DecoderFallbackException, as the following example illustrates.
Public Sub Main() Dim Bytes() As Byte Dim Enc As UnicodeEncoding Dim Value As String Bytes = NewBytes(&H0, &H20, &HD8, &H1, &H0, &H68, &HA7, &H0) Set Enc = NewUnicodeEncoding(True, True, True) On Error GoTo Catch Value = Enc.GetString(Bytes) Debug.Print CorString.Format("'{0}'", Value) Exit Sub Catch: Dim Ex As DecoderFallbackException Catch Ex, Err Debug.Print CorString.Format("Unable to decode {0} at index {1}", ShowBytes(Ex.BytesUnknown), Ex.Index) End Sub Private Function ShowBytes(ByRef Bytes() As Byte) As String Dim Byt As Variant Dim ReturnString As New StringBuilder For Each Byt In Bytes ReturnString.AppendFormat "0x{0:X2} ", Byt Next ShowBytes = Trim$(ReturnString.ToString) End Function ' The example displays the following output: ' Unable to decode 0xD8 0x01 at index 4
Public Sub Main() Dim Reader As StreamReader Dim Line As String Set Reader = NewStreamReader("TextFile.txt", Encoding.BigEndianUnicode) Line = Reader.ReadLine Do While Not CorString.IsNull(Line) Debug.Print Line Line = Reader.ReadLine Loop End Sub
The following example determines the number of bytes required to encode a character array, encodes the characters, and displays the resulting bytes.
Public Sub Main() Dim Chars() As Integer Dim U7 As Encoding Dim U8 As Encoding Dim U16LE As Encoding Dim U16BE As Encoding Dim U32 As Encoding ' The characters to encode: ' Latin Small Letter Z (U+007A) ' Latin Small Letter A (U+0061) ' Combining Breve (U+0306) ' Latin Small Letter AE With Acute (U+01FD) ' Greek Small Letter Beta (U+03B2) ' a high-surrogate value (U+D8FF) ' a low-surrogate value (U+DCFF) Chars = NewChars("z", "a", ChrW$(&H306), ChrW$(&H1FD), ChrW$(&H3B2), ChrW$(&HD8FF), ChrW$(&HDCFF)) Set U7 = Encoding.UTF7 Set U8 = Encoding.UTF8 Set U16LE = Encoding.Unicode Set U16BE = Encoding.BigEndianUnicode Set U32 = Encoding.UTF32 PrintCountsAndBytes Chars, U7 PrintCountsAndBytes Chars, U8 PrintCountsAndBytes Chars, U16LE PrintCountsAndBytes Chars, U16BE PrintCountsAndBytes Chars, U32 End Sub Private Sub PrintCountsAndBytes(ByRef Chars() As Integer, ByVal Enc As Encoding) Dim IBC As Long Dim IMBC As Long Dim Bytes() As Byte Debug.Print CorString.Format("{0,-30} :", Enc.ToString); IBC = Enc.GetByteCount(Chars) Debug.Print CorString.Format(" {0,-3}", IBC); IMBC = Enc.GetMaxByteCount(CorArray.Length(Chars)) Debug.Print CorString.Format(" {0, -3} :", IMBC); Bytes = Enc.GetBytes(Chars) PrintHexBytes Bytes End Sub Private Sub PrintHexBytes(ByRef Bytes() As Byte) Dim i As Long If CorArray.IsNullOrEmpty(Bytes) Then Debug.Print "<none>" Else For i = 0 To UBound(Bytes) Debug.Print CorString.Format("{0:X2} ", Bytes(i)); Next Debug.Print End If End Sub ' This code produces the following output. ' ' CorLib.UTF7Encoding : 18 23 :7A 61 2B 41 77 59 42 2F 51 4F 79 32 50 2F 63 2F 77 2D ' CorLib.UTF8Encoding : 12 24 :7A 61 CC 86 C7 BD CE B2 F1 8F B3 BF ' CorLib.UnicodeEncoding : 14 16 :7A 00 61 00 06 03 FD 01 B2 03 FF D8 FF DC ' CorLib.UnicodeEncoding : 14 16 :00 7A 00 61 03 06 01 FD 03 B2 D8 FF DC FF ' CorLib.UTF32Encoding : 24 32 :7A 00 00 00 61 00 00 00 06 03 00 00 FD 01 00 00 B2 03 00 00 FF FC 04 00


{% extends 'layouts/base.html' %} {% import 'macros/forms.html' as wtf %} {% block title %}Submit a Talk - {{ super() }}{% endblock %} {% block content %}


Welcome to Got IT Digital. After spending decades specializing in digital marketing and website maintenance for big brands, we decided to pool our experience and passion to create the all-encompassing digital firm that brands need.
We’re a specialized team that collaborates with your team to deliver high-performing results. Whether you’re new to the landscape of digital marketing or extremely well-versed and just need a helping hand, our team syncs with yours to build for growth and action.
Most importantly, we treat every job like it’s the most important job we’ve ever done.


There are millions (if not billions) of lines of code written for the Notes client in small, large, simple, complex, epidermal and business critical applications. With the rise of browsers, tablets and mobile devices there is a need to web and mobile enable these applications, not at least since IBM won't let the C code of the Notes client run on iOS (which it could).
Whether we like it or not, that body of code can be considered
technical debt.
The strategy I see is mostly the same: classify applications by size, business criticality and perceived complexity. Then retire as much as possible and start a pilot for one (either insignificant or overly complex - mostly both) application using XPages. From there it goes nowhere.
Guess what. There's a better way.
Keep the part where you decide if an application can be retired. If you need to retain read access you could check out
LDC Via, who remove the need to maintain a Domino infrastructure for those applications.
Then go and
collect evidence (I know
evidence based management isn't in fashion right now). There are elements in Notes that translate easily into web or mobile, parts that are hard and some stuff browsers can't do.
Collect how much you use of those, it makes decisions easier.
Design element and difficulty
- View navigation:
generally easy, its just data and Domino makes an excellent server for JSON data (I would render my own style, not the ?ReadViewEntries&Outputformat=JSON
, but that's another story)
- Categorised views:
medium. The twisties we got so used to in Notes are not a native construct on the web (show me one that was web native). But with a little effort you can devise interaction patterns, go peek here.
- View actions:
HARD. They can be anything. Worst when code for front-end and back-end classes get mixed
- Forms (read mode):
Easy to medium. You could simply open a form using a browser and see something - or craft some XPage to show
- Forms (edit mode):
medium to hard (see below): You need to get your validation right and all the nice Hide-when conditions
- RichText:
HARD to impossible. RichText != HTML
(and that's not even !==
). For rendering you want to have Ben's enhancements as a start. RichText can contain attachments, wild formatting, OLE objects, Sections, Hidden stuff, tabbed or timed tables etc. All that doesn't really translate to HTML. To really know how RichText was used in your application and organisation you need to scan the actual data and analyse how RichText was used in each application
- Code in forms:
HARD. There is no LotusScript running in your browser, so you need to rewrite that logic. So better to know how many functions and lines of code you are looking at. Do your CoCoMo analysis. Code in fields (on enter/exit) needs to be re-though (nice word for retired)
- Attachments:
TEDIOUS. HTML doesn't know attachments, so you have to bridge between user expectations: can put an attachment anywhere in the text and the storage reality of HTML: simulate this by placing an image and a link. You might consider adding webDAV for a round trip editing experience
- Reader & Author fields:
Easy. Works
- Encryption and Signature:
(popular in approval applications) wait for Domino 9.0.2 - it supposedly will work there
- Business Logic in hide-when formula:
hard to find in Domino designer, DXL to the rescue
This list is probably not complete, but you get the gist
The approach
I would advocate (after the retirement of old applications):
- Look at the whole application portfolio, not just one pilot. The best analogy: remember how long it took to drive your first 100m in a car (and I'm not talking about the 18 years wait to come of age). You had to take driving lessons (at least in civilized countries you have to), learn for the test, get a car (bargin with a parent?). So it took you around 20-40h, lets say: 20h. So you conclude to get to the supermarket, 3 km away it will take you 600h - so driving is out, you rather walk. Same applied to the "pilot". The first application is the hardest, so your experience wih it will screw the estimations.
Furhermore looking at all applications gives you new synery potential and efficiency of scale
- Create one module that handles view navigation for all applications (that probably would be a OSGi plug-in)
When initially opening a document, it would point to a notes:// URL (you could have a little icon indicating that). Within a few weeks you have a working application giving some access to your Notes data (I would use JSON, AngularJS and Ionic for that app)
- Next generate the forms using angular-formly or some {{mustache}} templates, so you have read-only access (DXL and XSLT skills come in handy)
- Add a "request" functionality that lists the functions (Action buttons) an application once had, let users propose "I want this", so you get some priorities sorted out
- Transit to contemporary development with a Domino backend
- Do not fall into the trap: must look and work like a Notes client (we had that with the Java applets in R5 - last century). If someone is happy with the look of the Notes client - give them the IBM Client access plugin (PC only)
Why no magic button?
You could ask: why doesn't IBM provide this as an automatic module? After all they wrote LotusScript, the formula language and the Notes client. The short answer: IBM tried, we even had a prototype of LotusScript running on the JVM. The Notes client is such an incredible forgiving environment, any quality and mix of code "simply runs"™. I've seen code happily running (yes I'm talking about you 8000-lines-function) that was a melange of unspeakables.
Browsers on the other hand are fragile and unforgiving (not talking about
quirks mode here). Any attempt to automate that never reached the stage of "point-and-shoot" - so it will not become an offering.
Why not just default and rewrite?
Good idea. Until you look closer. Domino's
declarative security (ACL, Reader and Author fields anyone) is hard to replicate. The schema free Domino NoSQL store does neither map to Sharepoint nor an RDBMS (CouchDB could work), so you suddenly deal with data migration - a nightmare that can eat 80% of your total project cost. Lastly, the incremental approach outlined here allows to continue to use the applications (and eventually enhance them along the way), so disruption is minimised. Also the working Notes client provides a fallback for missing or malfunctioning features, this substantially lowers application risks.
Business partners to the rescue
You are not alone on this journey.
IBM business partners and
IBM Software Services for Collaboration are here to help. If you like the approach "I see Notes as one huge pool", you might want to talk to
Redpill, they perfected this approach.
As usual YMMV.


{% load static %}
Welcome to ContestSpace ! Participating virtually in past Codechef contests is just a step away. Go ahead and participate individually or in a team. Wish you all the luck!


Mean Creek
Mean Creek is a movie starring Rory Culkin, Ryan Kelley, and Scott Mechlowicz. When a teen is bullied, his brother and friends lure the bully into the woods to seek revenge.


Akan names derived from Ghanian culture
Get your Akan Name
Find out what your Akan name is, proceed to filling the form below and click the submit button.
Year of Birth:
Month of Birth:
Day of Birth:
Select your gender:
..select..
Male
Female
Submit
You were born on:
Your Akan name is:


DACA recipient arrested for human smuggling
by Cydeni CarterTUCSON, AZ — A Deferred Action for Childhood Arrivals (DACA) recipient has been arrested for smuggling an illegal man near Lukeville, Arizona Tuesday night.
U.S. Border Patrol agents out of Tucson caught a 29-year-old woman, attempting to smuggle a 21-year-old man in the inspection lane at the State Route 85 immigration checkpoint.
The DACA recipient was caught in a secondary inspection at around 11 p.m. Tuesday. Agents determined the driver, a Colorado resident, to be smuggling the passenger across the border, a Mexican national who was illegally in the United States.
The Kia sedan was seized by Border Patrol agents and the woman is now custody with human smuggling charges. The illegal alien man has been processed for violation of immigration laws.


{% extends "base.html" %} {% block title %}Reset Password{% endblock %} {% block content %}
Please provide your email address to receive instructions for resetting it


| Tournament | Score |
|---|---|
| {{score.tournament.name}} | {{score.total}} |
| Tournament | Player O | Player X | Winner | |
|---|---|---|---|---|
| {{match.tournament.name}} | {{match.player_one.class_name}} | {{match.player_two.class_name}} | {{match.current_winner}} | details |
| {{match.tournament.name}} | {{match.player_one.class_name}} | {{match.player_two.class_name}} | {{match.current_winner}} | details |


Canadian R&B Singer Tamia Releases Festive New Single ‘Someday At Christmas’
by Brent FurdykAs the holidays approach, Canadian R&B singer Tamia is celebrating the season with the release of her festive new single “Someday At Christmas”.
The latest from the six-time Grammy nominee is a cover of the holiday classic originally recorded by Stevie Wonder.
The song clearly has a lot of meaning for the Ontario native, and in a recent interview with the Los Angeles Times she discussed why she thinks it’s important to have a special connection with a song in order to record it.
“Well, I never record any song that I don’t love or like,” explained the 44-year-old singer-songwriter. “Some I fall in love with more than others, but I don’t go into the studio and just record songs all over the place. If I don’t like the song or if I don’t believe in it, I don’t record it. The toughest is going over the songs you love and seeing what speaks to you in the moment. If they don’t make a cohesive album, I hold on to them.”
“Someday At Christmas” is available on all digital platforms right now.