Polyglot Storage Lab
Document
In this scenario based HOL, you will learn how to build a
‘polyglot persistence’ data pattern that is common in modern cloud hosted
applications. Requirements of modern applications, such as, greater scale
and availability, have driven the industry to begin using a much broader range
of technologies for storing data within an application. Microsoft Azure
provides a range of storage technologies that support these architectures and
this HOL provides an example of the use of these in the well understood
scenario of e-Commerce. With data services in Microsoft Azure, you can quickly
design, deploy, and manage highly-available apps that scale without downtime
and that enable you to rapidly respond to security threats. Features built into
services like Azure SQL Database, Azure Search, and Azure DocumentDB help your
apps scale smartly, run faster, and stay available and secure.
In this HOL you will see a browser based e-commerce application running under the LCA approved sample company name ‘AdventureWorks’. It has been created to demonstrate functionality provided by the following data storage technologies (SQL Database, DocumentDB, Search, Table Storage). In a real application, decisions will need to be made as to where data is stored. In this HOL we wish to highlight; how using multiple Azure data service technologies allows you to take a modern approach to data in your applications.
Note- The website (which gets built out of this HOL) is not intended to be a fully functioning site. It is not designed to be a reference e-commerce implementation nor a starting point for a customer’s implementation of an e-commerce site on Azure, rather it will provide the following functionality in order to demonstrate the selected storage technologies.
In the course of this lab, you will gain greater familiarity with Azure SQL Database, DocumentDB, Azure Search and Table Storage through performing the following tasks:
·
Familiarize yourself with one of the tenant-company’s websites and
its Azure SQL Database backend.
·
Create a new database using the Azure portal.
·
Configure and implement vertical scaling by increasing the
capacity of a database.
·
Use Azure SQL Database auditing features to track down an
erroneous deletion from a database.
·
Use Azure SQL Database point-in-time restore to correct the
deletion (Optional)
·
Configure and implement Azure SQL Database geographic disaster
recovery to prevent large-scale data loss.
·
Locate data using Azure Search.
·
Modernize and create an iterative experience using DocumentDB.
·
|
Demo Story
Elements |
Completed (Yes/No) |
|
Scenario 0: Azure Search – Index
Definition Discuss how we maintain a product
index within Azure Search. Contained within this index, we store everything
to show you the products on this home page. Setting up a new index and
getting started is simple. |
|
|
Scenario 1: Azure Search –
Indexing Documents Discuss how we index documents to
Azure Search, and how we can define indexers to do this for us. |
|
|
Scenario 2: Azure Search – Search
Suggestions Discuss how we use out of the box
Azure Search to provide search suggestions to the user. |
|
|
Scenario 3: Azure Search - Facets Discuss we use Facets provided by
Azure Search to provide us with refiner items. |
|
|
Scenario 4: Azure Search –
Scoring Profiles Discuss we can use scoring profiles
to manipulate the priority of search results. |
|
|
Scenario 5: DocumentDB Discuss how we can rapidly
produce new applications using DocumentDB. |
|
|
Scenario 6: DocumentDB – Schema
Free Query Discuss how DocumentDB is well
suited to query across schema free attributes. |
|
|
Scenario 7: DocumentDB- Nested
Documents/Consistency Discuss how DocumentDB supports
tunable consistency. |
|
|
Scenario 8: DocumentDB -
Transactions Discuss how DocumentDB provides
transaction processing over multiple documents. |
|
|
Scenario 9: Table Storage Discuss Azure Table storage. |
|
|
Scenario 10: Azure SQL –
Transaction Data Store Discuss how we can use Azure SQL
to store transactional records. |
|
|
Scenario 11: Azure SQL –
Predictable Performance Discuss how we can easily upscale
or downscale the performance of Azure SQL. |
|
|
Scenario 12: Azure SQL – Database
Recovery Discuss how Azure SQL
automatically stores Point in Time restore points, and how we can leverage
them to undo mistakes. |
|
|
Scenario 13: Azure SQL - Auditing Discuss how we can store the
audit log to an Azure Storage account. |
|
|
Scenario 14: Azure SQL - Vertical Scaling Discuss how the website is
anticipating a big sale and how that might require some increased capacity.
Simulate some load using the mass order generator, and then show the spike in
resource utilization. From the portal, show how easy it is to scale up by
changing the size of an instance (for example, from B1 to S3) and how quickly
the change is implemented. Re-run the mass order generator and show how the
increased capacity has significantly improved the site’s ability to handle a
spike in sales. |
|
|
Conclusion/Summary |
|
|
TOTAL |
|
|
Screen |
Click
Steps |
|
|
DEMO SETUP |
|
|
Open Browser to http://inbuildxxxx.azurewebsites.net (inbuildxxxx is what you used in the deployment. “in” is your initals
and buildxxxx is you azure pass ID). |
|
|
·
Select Melissa
MacBeth as the customer by o clicking on the person Icon o selecting Melissa MacBeth |
|
Screen |
Click Steps |
Demo Script |
|
|
Scenario 0: Azure Search – Index
Definition |
|
|
|
1.
Open browser http://inbuildxxxx.azurewebsites.net |
|
|
|
2.
Open a new browser window and navigate to http://jsonviewer.stack.hu/#http://inbuildxxxx.azurewebsites.net/Api/Discover/AzureSearchSchema/Products 3.
Click Viewer in
the top left hand corner 4.
Expand Out
Fields Collection |
This
is a JSON that defines the schema of our search index for Products. |
|
|
5.
Navigate to http://inbuildxxxx.azurewebsites.net/Api/Discover/Index |
Let’s
see how we create the index. This
is a JSON representation of the Index object that is created in the
AdventureWorks code. This is then passed into the Azure Search Client. |
|
|
Build
Index creates the index object as seen above. This is then passed into
CreateAsync. This
index is created by using some custom attributes on our models, which then
builds up the index object to pass to Azure Search. https://msdn.microsoft.com/en-us/library/azure/microsoft.azure.search.searchserviceclient.aspx Source
Code: MSCorp.AdventureWorks.Core->AzureSearchClient.cs->Line 80 |
|
Screen |
Click Steps |
Demo Script |
|
|
Scenario 1: Azure Search – Indexing
Documents |
|
|
|
1.
Navigate to the Search Box. |
|
|
|
2.
Type Mountain into the search box and press enter |
You
will notice that there currently are no products being displayed on the
website. We
need to index some documents to the search index. There are two ways to do
this. |
|
|
We
can upload documents as and when we need to, we are passing an object that
represents the document and then indexing it through the Azure Search Client. You
can either use a full model (class) which matches your index type. Or you can
instantiate a Document object which allows you to build up the document with
key value pairs. (This allows for partial updates where you only need to
update one field). To
do an update. You would call indexClient.Documents.ReplaceAsync. The key must
be provided in the object. But
there is a more automated way, where no code is required. Source
Code: MSCorp.AdventureWorks.Core->AzureSearchClient.cs->Line 200 |
|
|
|
3. Navigate to http://inbuildxxxx.azurewebsites.net/Api/Discover/Indexer |
We
can create an indexer which is scheduled task that will look at a datasource
and every x minutes and index them into the Search Index. To
create the Indexer. You must first define a datasource, and then define an
indexer that uses that data source. This
returns a JSON that represents the DataSource and Indexer objects used in
AdventureWorks for the products indexer. |
|
|
4. Navigate to http://inbuildxxxx.azurewebsites.net/Indexer/Create 5. Navigate to http://inbuildxxxx.azurewebsites.net/Indexer/Run |
Let’s
create the Indexer by running this command. First we are creating it. Then we
are setting it to run. Note:
This is creating a predefined indexer which will link the existing DocumentDB
database for products with the existing azure search service. |
|
|
While
we wait for the Indexer to run, let’s observe how we created the data source
and indexer. BuildDataSource
creates a DataSource object (as shown in step 3). This is a datasource that
looks at the DocumentDB product database. BuildIndexer
creates an Indexer object (as shown in step 3). After
a couple of minutes, we should start getting results (shown in Scenario 2). Source
Code: MSCorp.AdventureWorks.Core->AzureSearchClient.cs->Line 114 |
|
Screen |
Click Steps |
Demo Script |
|
|
Scenario 2: Azure Search – Search
Suggestions |
|
|
|
1.
Navigate to the search box. |
|
|
|
2.
Type ‘Mountian’ as a misspelling of ‘Mountain’ into
the search box. (Don’t press enter
yet) |
We
use out of the box Azure Search to provide search Suggestions to the
user. Suggestions is the term Azure Search uses to retrieve suggestions based
on partial search input. They can only be set for fields of
type Edm.String or Collection(Edm.String) One
of the options for Search suggestions is fuzzy which takes a Boolean
parameter. When
fuzzy is turned on the Search service will find suggestions even if there's a
substituted or missing character in the search text. Worth
noting that while this provides a better experience, in some scenarios it
comes at a performance cost as fuzzy suggestion searches are slower and
consume more resources. |
|
|
3.
Open a new browser window and navigate to
http://jsonviewer.stack.hu/#http://inbuildxxxx.azurewebsites.net/Api/Search/Suggest/Mountian
- Notice that we are sending the misspelling of mountain to the Suggest
endpoint of the service. 4.
Expand out the element 0 of the Results collection. |
See
that we are getting the correct spelling of mountain when passing in the
incorrect spelling. |
|
|
Let's
see how this suggest was executed. Notice
UseFuzzyMatching = true. This is what allows misspellings to still be found. Source
Code: MSCorp.AdventureWorks.Web->ApiSearchControllert.cs->Line 48 Source
Code: MSCorp.AdventureWorks.Core->AzureSearchClient.cs->Line 300 |
|
Screen |
Click Steps |
Demo Script |
|
|
Scenario 3: Azure Search – Facets |
|
|
|
1.
Navigate to the search box. 2. Type ‘Mountain’
into the search box and press Enter. |
Azure
search service provides us both the Search results and the refiner lists you
see on the left hand side of the screen. Notice that we get other products
which contain the word ‘Mountain’, these can be filtered using Azure Search
Facets. |
|
|
3. Select these
Facets (Filters) ·
Black ·
40” |
As
we select and un-select facets, we are going back to Azure Search service for
the facet filtering including the text search passed (in this case Mountain). We
do this by filtering against the facets using the word values and piped
filters E.g.
facet=color,values:Black|Silver|White |
|
|
4. Open a new
browser window and navigate to http://jsonviewer.stack.hu/#http://inbuildxxxx.azurewebsites.net/Api/Search/Products/Mountain |
Here
you can see the Facets collection which provides us with our “groups” of
filters that we can use to refine the results. (We provided the facet
collections we wanted in Scenario 2.) |
|
|
Let's
see how we can use Facets with Azure Search. See
that we have provided a Filter with this search. In this case we are asking
to match where the product category is equal to the one passed into the
function. Source
Code: MSCorp.AdventureWorks.Web->ApiSearchControllert.cs->Line 176 |
|
Screen |
Click Steps |
Demo Script |
|
|
Scenario 4: Azure Search – Scoring
Profiles |
|
|
|
1.
Navigate to http://inbuildxxxx.azurewebsites.net/Search/Mountain (or
type mountain in the search box and click enter) 2.
Click a product image which is not at the top of a
list. (but has mountain written in the name) 3.
Click Edit Product Details. |
This
will open up a new browser window from here to review the product definition.
When looking at product detail page you are viewing the full product
representation which is stored in DocumentDB |
|
|
4. Update the
product priority to be significantly larger than it is right now and click
save. Note: the priority scoring profile has been setup for values
between 1 and 10,000. |
As
the product is updated we persist the changes into product storage and then
update the representation within Azure Search. |
|
|
5. Navigate back to
the original browser window with the search results and refresh the page. |
By
updating the products priority we have altered the order in which the
products are returned. Note: This may take 5 minutes. Or we can force
it straight away by running http://inbuildxxxx.azurewebsites.net/Indexer/Run |
Screen |
Click Steps |
Demo Script |
|
|
Scenario 5: DocumentDB |
|
|
|
1. Navigate to http://inbuildxxxx.azurewebsites.net/Product/BK-M18B |
In
AdventureWorks we have two product stores, DocumentDB which is the master
product store and the product index from within Azure Search. The product
information which you see on this product details page is surfaced directly
from DocumentDB and passed to the browser for display. |
|
Screen |
Click Steps |
Demo Script |
|
|
Scenario 6: DocumentDB – Schema
Free Query |
|
|
|
1. Navigate to http://inbuildxxxx.azurewebsites.net/Product/BK-M18B 2. View the
specifications of the product. |
Each
product has some level of specifications which will be unique to each
product. E.g. Bike “Frame Material : Carbon Fiber” VS Water bottle “Material
: Eastman Tritan copolyester” |
|
|
3. Open a new
browser window and navigate to http://jsonviewer.stack.hu/#http://inbuildxxxx.azurewebsites.net/Api/Search/Product/BK-M18B 4. Expand out the
Product Attributes Collection. |
These
specifications show the schema-free nature of the DocumentDB, this is not
particularly special but next we will talk about query over schema-free data
which really separates DocumentDB out from the competition. |
|
|
5. Go back to the
original page - http://inbuildxxxx.azurewebsites.net/Product/BK-M18B 6. Click on the text
‘mountain biking’ under the specification ‘recommended for’ |
In
other areas of the application, we’ve used Azure Search service for
predictable searching across known product attributes E.g. name, description,
category, and manufacturer. DocumentDB is well suited to query across schema
free attributes. DocumentDB provides a Rich query over schema-free design
which allows us to query over these product specifications. Specifically: ·
Schema-free design allows for agile development and
continuous iteration ·
Differentiated querying through no limitation to the
number of ways you can query your data because there is no forced,
pre-defined set of indices ·
Query hierarchical JSON data through a succinct
query grammar, including transforms and inline JavaScript |
Screen |
Click Steps |
Demo Script |
|
|
Scenario 7: DocumentDB – Nested
Documents / Consistency |
|
|
|
1. Navigate to http://inbuildxxxx.azurewebsites.net/Product/BK-R50B 2. Add a comment
underneath the existing review and click add. |
The
product review document holds the original review and any related comments
are stored as sub documents. This makes up the hierarchy. |
|
|
3. Navigate to http://jsonviewer.stack.hu/#http://inbuildxxxx.azurewebsites.net/Api/Search/ProductReview/BK-R50B |
Looking
at the JSON you see that the responses collection has a set of comments
displaying the nested document structure. |
|
Screen |
Click Steps |
Demo Script |
|
|
Scenario 8: DocumentDB –
Transactions |
|
|
|
1. Navigate to http://inbuildxxxx.azurewebsites.net/Admin/Product/Discount 2. Select
Recommended For in the first drop down. 3. Select Mountain
Biking in the second drop down. |
Now
that we have found all the products which we want to update let’s give all
the products a 15% discount. |
|
|
4. Type 15 into the
set discount. 5. Click Apply
Discount (Wait for confirmation). |
Under
the covers we are updating all the product documents with that discounted
price within a single ACID transaction. DocumentDB provides a full-featured
NoSQL document database service including out of the box transactional
processing over multiple documents through application defined j JavaScript
/ stored procedures. |
|
|
6. Right click any
product in the list and Open in a new tab. 7. Click the Edit
Product Details. 8. Save the product
and go back to original discount tab. (Do not refresh). |
|
|
|
9. Click Apply
Discount (wait for confirmation). |
See
that the update discount has failed. Because we edited one of the products in
the set, it was unable to be updated as it was out of date. We
have leveraged the ETag on the product, using a concurrency check we have
demonstrated one scenario where we wanted to abort the underlying
transaction. We
achieve this behavior through further use of stored procedures. Stored
procedures, triggers and UDFs are modelled after the concepts supported by
RDBMS. All JavaScript logic is executed within an ambient ACID transaction
with snapshot isolation. During the course of its execution, if the
JavaScript throws an exception, then the entire transaction is aborted. |
|
|
10. Navigate to http://inbuildxxxx.azurewebsites.net/Api/Discover/StoredProcedures/ProductCollection 11. Scroll to
procedure ApplyDiscountToProducts. |
This
is the stored procedure responsible for apply discounts and checking the
Etag. |
|
Screen |
Click Steps |
Demo Script |
|
|
Table Storage |
|
|
|
Scenario 9: Table Storage |
|
|
|
1. Make sure you are
signed in as Jeff Wang. 2. Navigate to http://inbuildxxxx.azurewebsites.net/Product/BK-M18B 3. Click Add to
Cart. |
|
|
|
4. Click on the cart
icon. |
|
|
|
5. Navigate to http://jsonviewer.stack.hu/#http://inbuildxxxx.azurewebsites.net/Api/Cart/108 |
As
you can see by the representation of what is in the shopping cart, we are
storing an un-schematized JSON representation of the order. If the order
schema changes in any we don’t need to consider changing anything about table
storage. All information pertaining to the information is stored in a single
document, so no links or hierarchy are required. We
store the shopping cart based on the customer who has added the order to the
cart. All the information is stored in a single document so no links or
hierarchy is required. Table
storage is cheap and accessible as a simple RESTful service. The shopping
cart is a canonical use of key value pair - simple to retrieve based on the
customer key and not required after the users session is completed. |
|
Screen |
Click Steps |
Demo Script |
|
|
Azure SQL |
|
|
|
Scenario 10: Azure SQL –
Transaction Data Store |
|
|
|
1. Navigate to http://inbuildxxxx.azurewebsites.net/Checkout 2. Click Make
Purchase |
As
part of making the purchase we take what is in the shopping cart and push it
into an Azure SQL database as the transactional store. |
|
|
3. Open up visual
studio and add a data the connection to the database (Azure SQL Database
Instance), then open up a T-SQL query window |
Let’s
head over to the database and validate that we see a new order in the
database. Run
the following SQL looking at the order and lines. |
|
|
4. Execute these
queries SELECT *
FROM [dbo].[Orders] ORDER BY Id DESC SELECT * FROM [dbo].[OrderLines] ORDER BY Order_ID
DESC |
By
using Azure SQL, we remove virtually all infrastructure maintenance with SQL
Database which provides automatic software patching as part of the service.
We are also able to leverage SQL Server skills across on-premises and cloud
environments with a familiar relational foundation and T-SQL functions. These
two aspects of Azure SQL make it both familiar and self-managed which have
enabled AdventureWorks as a company to drive fast time-to-market and
unprecedented efficiencies with near-zero maintenance service. |
|
Screen |
Click Steps |
Demo Script |
|
|
Scenario 11: Azure SQL –
Predictable Performance and Vertical Scaling |
|
|
|
1. Login to the
Azure portal at https://portal.azure.com/ with credentials
which has access to the database hosting the application. 2. Click Browse
and select SQL databases. 3. Select the AdventureWorks,
Click on Monitoring in the Database blade. |
As
you can see from the monitoring on the AdventureWorks database, at the moment
with the current load we are in no way stressing the database. The database
is configured on the premium service tier. This is exactly where we want to
be with the sale event starting this weekend. But it was a different story
last year. |
|
|
4. Navigate to http://inbuildxxxx.azurewebsites.net/images/LastYearMonitoring.jpg |
Here
are the metrics from last year’s Black Friday sale event. We were caught out
with the amount of traffic generated from so many customers placing orders. We
can see that we were reaching to the 80% for Log writes per second and the
CPU percentage. Based on these telemetry results we can clearly see our
database was getting close to the upper threshold limit of the service tier,
we had to quickly react and upgrade to a higher performance level to maintain
great performance and thus a great customer experience. This
was a good problem to have, but this year we are prepared in advance as we
have already scaled our database service tier up to premium. |
|
|
5. Close the tab and
navigate back to the open portal window, Right click on the graph and select
edit chart. |
The
Azure Portal has built in dashboard views showing telemetry data. This allows
us to get an understanding of what’s been happening on our databases over
time. Note as developers we also have the ability to programmatically obtain
this. We
can easily include or exclude metrics to plot on the graph and change the
charts time range. Options for telemetry are Storage, CPU percentage,
Physical Data reads, Log writes per second, blocked by firewall, deadlocks,
failed connections and successful connections. |
|
|
6. Click on
Monitoring Graph. |
Notice
when I click on the monitoring graph we have the ability to add an alert. We
can configure alerts on performance metrics. For example, if you expect the
workload on your database to grow, you can chose to configure an email alert
whenever your database reaches 80% on any of the log writes percentage. You
can use this as an early warning to figure out when you might have to switch
to the next higher service tier. Alternatively you could use it to monitor
when it falls below a certain percentage to downgrade the service tier. Let's
do an experiment. |
|
|
7. Navigate to http://inbuildxxxx.azurewebsites.net/MassOrdering 8. Click Generate
and Save Mass Order Data (and wait). |
This
will generate 1,000 orders simulating a big sale event with big usage spikes. You
will see the time it took for this process. |
|
|
9. Go back to the
monitoring graph. |
Note:
This may take 5-10 minutes to update. Notice
that DTU percentage is at 20% (your results may differ slightly). We
will compare the results when we scale down in Scenario 19. |
|
Screen |
Click Steps |
Demo Script |
|
|
Scenario 12: Azure SQL – Database
Recovery |
|
|
|
1. Sign in to the Azure Management Portal using your
Microsoft account. 2. From the Databases list,
select the database you want to restore - Only the new service tiers
Basic, Standard and Premium service tiers support Point in Time. 3. At the bottom of
the page in the command bar, click Restore. This launches
the Specify Restore Settings dialog box. 4. Select the Restore
Point. 5. Specify a new
database name in the Database Name field. 6. Click create to
submit the restore request. |
Point
in Time Restore is designed to recover a database to a specific point in time
within the backup retention period supported by the service tier of the
database. Restoring creates a new database with the same service tier that
was in use at the chosen restore point and the lowest performance level
supported by that tier. For example, if you restore a database which was set
to Premium at the specified restore point in time, the new database will also
be a Premium database. The Restore
Point is used to specify the point in time to which your database should
be restored. The restore point must be within the retention period supported
by the database service tier and the Point in Time setting must be set to ON. The
backup retention period is a maximum of seven days for Basic, 14 days for
Standard, and 35 days for Premium. Requests to restore to a point in time
outside of the supported retention period for the service tier will fail. Restore
operations may take a long time to complete. You can monitor the status of
the restore operation on the Databases list. Databases that are
being restored are visible in the Databases list once the restore
is in progress. The row is populated with the database settings and the
status of the restore operation which is displayed as Restoring until
the operation is completed. |
|
|
7. Navigate to the
SQL Server. 8. On the side tab,
scroll down to deleted databases. |
We
can also restore a deleted database. This list includes a list of Restorable
Deleted Databases available to restore. Similarly to above, the
restore point must be within an appropriate retention period supported by the
service tier. Point
in Time Restore enables restoring a database to a point in time in the past
within the retention period for the given service tier. However, the
retention period for the service tier may not be long enough to meet business
needs. In this case, consider creating an archived database using methods
such as automated export through Azure. Point
in Time restore offers self-service control over data restoration from
available backup data, putting the power to restore in the hands of customers
in the case of a human or programmatic data deletion scenario. Using
Azure there are other options for database recovery depending on the tier you
are using. We’ve already mentioned Automated Export above, but there is also
Geo restore, standard Geo replication and a database copy and manual restore.
|
|
Screen |
Click Steps |
Demo Script |
|
|
Scenario 13: Azure SQL – Auditing |
|
|
|
1.
Sign in to the Azure
Management Portal using your Microsoft
account. 2.
Select a database from
the Databases list. 3.
Click Auditing and Threat
Detection. |
This
will enable the auditing and launch the auditing configuration blade. |
|
|
4.
In the auditing
configuration blade, select the Azure storage account where logs will be
saved. |
If
we had several audited databases, we would use the same storage account for
all audited databases to get the most out of the preconfigured reports
templates. Audit
logs are aggregated in a single Azure Store Table
named AuditLogs in the Azure storage account. |
|
|
5.
Under Auditing Event,
click All, then click OK. |
Depending
on the requirements for auditing, we can configure what type of events are
audited. Access to data, Schema changes (DDL), data changes (DML), Accounts,
roles, and permissions (DCL), Security exceptions. We
can also save this configuration as default. This means that these settings
would apply to all future databases on the same server, and any that don't
already have auditing set up. You can override the settings later for each
database by following these same steps. |
|
|
6.
Click Save. |
|
|
|
7.
Click Explore. 8.
Click Open in Excel. |
To
use the template on your audit logs, you need Excel 2013 or later, and Power Query.
The template here has fictional sample data in it which we can use to show
you the powerful capabilities. Let’s
have a look through some of the sheets. Orientation: Documentation links and brief description of the
visible interactive reports Anomalies: Events that usually should not appear in normal
operation. Double-click a Count value for the list of events Drill Down: Ability to slice to the desired specific set of
rows. Double-click a Count value for
the list of events Event Type Distribution: Distribution of Audit Logs
Event Types by several dimensions: Database, Month --> Day, Weekday, Hour
in the Day Event Time Analysis: Analysis over time by
Event Type. The two vertically adjacent charts enable correlating events with
different scale of appearance. Double-click a count value for the list of
events With
the Database Auditing capability, Microsoft Azure SQL Database offers a tool
to streamline compliance-related activities, gain knowledge about what is
happening in your database, and identify trends, discrepancies, and
anomalies. These can serve to enhance business visibility, or potentially
indicate business concerns or suspected security violations. |
Screen |
Click Steps |
Demo Script |
|
|
Scenario 14: Azure SQL - Vertical Scaling |
|
|
|
1. Login into the
Azure Portal. 2. Select SQL
Databases and select the AdventureWorks database. 3. Click on the
pricing tier tab. |
|
|
|
4. Click on Basic
then click Select. |
This
will downgrade our database to five DTUs. |
|
|
5. Navigate to http://inbuildxxxx.azurewebsites.net/MassOrdering 6. Click Generate
and Save Mass Order Data and wait. |
This
will generate 1,000 orders, simulating a big sale event with big usage
spikes. You
will see the time it took for this process. |
|
|
7. Go back to the
monitoring graph (Database Page). |
Note:
This may take 5-10 minutes to update. Notice
that DTU percentage is at 98%. Your results may differ slightly. This
has drastically reduced the performance of the SQL Server. Luckily with Azure, it is easy to vertically
scale the services on an adhoc or full time basis. |


Find text in the tracked files
git grep
git grep <TREEISH>
Start the process of finding a commit where tests broke
git bisect start
Mark current point as bad
git bisect bad
Mark an older point as good
git bisect OLDERTREEISH
If you want to manually mark the status of the current checkout and proceed to the next, type either of the following:
git bisect bad
git bisect good
Show the results of the bisect
git bisect log
Start the bisect over again or to finish the process
git bisect reset
If you wish to automate the run of the tests and marking boolean outcome (testing), use the run
command. The program must return 0 for good
, 125 for skip
, 1-127 (except 125) for bad
, and any negative value like -1 for abort
.
git bisect run <COMMAND>
Diagram the result of testing
git bisect visualize


Web Development Resources Part 3
Web Development Resources Part 3
I’m the psychological equivalent of a physical hoarder only instead of empty soda cans and dead racoons it’s lists of links and resources.
Here’s parts one and two:
Performance
- https://developers.google.com/web/fundamentals/performance/webpack/
- https://iamakulov.com/notes/resize-scroll/
- https://iamakulov.com/notes/optimize-images-webpack/
- https://developers.google.com/web/fundamentals/performance/webpack/decrease-frontend-size
- https://code.facebook.com/posts/557147474482256
- www.phpied.com/rendering-repaint-reflowrelayout-restyle
- https://blogs.windows.com/msedgedev/2017/03/08/scrolling-on-the-web/
- https://developers.google.com/web/fundamentals/performance/lazy-loading-guidance/images-and-video/
Case studies
Interviews
- https://performancejs.com/post/hde6d32/The-Best-Frontend-JavaScript-Interview-Questions-(Written-by-a-Frontend-Engineer)
- https://github.com/nishant8BITS/123-Essential-JavaScript-Interview-Question
- https://github.com/yangshun/tech-interview-handbook
- https://github.com/IgnaciodeNuevo/frontend-development-interviews#questions-to-know-the-company-better
- http://www.creativebloq.com/features/12-common-javascript-questions-answered
- http://www.thatjsdude.com/interview/js1.html
- http://www.thatjsdude.com/interview/js2.html
- http://www.thatjsdude.com/interview/dom.html
- https://github.com/khan4019/front-end-Interview-Questions
- https://github.com/h5bp/Front-end-Developer-Interview-Questions
Web
Architecture
- https://medium.com/@cramforce/designing-very-large-javascript-applications-6e013a3291a3
- https://www.toptal.com/javascript/comprehensive-guide-javascript-design-patterns
- https://addyosmani.com/resources/essentialjsdesignpatterns/book/
Accessibility
Node
JS
- https://blog.mgechev.com/2018/01/29/javascript-decorators-aop-autobind-memoization/
- https://developers.google.com/web/fundamentals/primers/async-functions
- https://developers.google.com/web/fundamentals/primers/promises
- https://www.toptal.com/javascript/asynchronous-javascript-async-await-tutorial
- https://www.toptal.com/software/definitive-guide-to-datetime-manipulation
- https://www.toptal.com/javascript/javascript-es6-cheat-sheet
- https://www.safaribooksonline.com/library/view/es6-in-motion/10000MNLV201701/
CSS
If you found this guide helpful feel free to checkout my GitHub/gists where I host similar content:
bgoonz’s gists
Instantly share code, notes, and snippets. Web Developer, Electrical Engineer JavaScript | CSS | Bootstrap | Python |…gist.github.com
Instantly share code, notes, and snippets. Web Developer, Electrical Engineer JavaScript | CSS | Bootstrap | Python |…gist.github.com
bgoonz — Overview
Web Developer, Electrical Engineer JavaScript | CSS | Bootstrap | Python | React | Node.js | Express | Sequelize…github.com
Web Developer, Electrical Engineer JavaScript | CSS | Bootstrap | Python | React | Node.js | Express | Sequelize…github.com


|
Little Heroes
|
Extended progress bar that has backwards compatibility logic and adds interaction support. More...
Protected Member Functions | |
| override void | Upgrade () |
| Upgrade from legacy functionality. More... | |
| override void | OnStart () |
| Register an event listener. More... | |
| void | OnPressBackground (GameObject go, bool isPressed) |
| Position the scroll bar to be under the current touch. More... | |
| void | OnDragBackground (GameObject go, Vector2 delta) |
| Position the scroll bar to be under the current touch. More... | |
| void | OnPressForeground (GameObject go, bool isPressed) |
| Save the position of the foreground on press. More... | |
| void | OnDragForeground (GameObject go, Vector2 delta) |
| Drag the scroll bar in the specified direction. More... | |
| void | OnKey (KeyCode key) |
| Watch for key events and adjust the value accordingly. More... | |
| Protected Member Functions inherited from UIProgressBar | |
| void | Start () |
| Register the event listeners. More... | |
| void | Update () |
| Update the value of the scroll bar if necessary. More... | |
| void | OnValidate () |
| Invalidate the scroll bar. More... | |
| float | ScreenToValue (Vector2 screenPos) |
| Drag the scroll bar by the specified on-screen amount. More... | |
| virtual float | LocalToValue (Vector2 localPos) |
| Calculate the value of the progress bar given the specified local position. More... | |
| void | SetThumbPosition (Vector3 worldPos) |
| Set the position of the thumb to the specified world coordinates. More... | |
Protected Attributes | |
| bool | mInverted = false |
| Protected Attributes inherited from UIProgressBar | |
| UIWidget | mBG |
| UIWidget | mFG |
| float | mValue = 1f |
| FillDirection | mFill = FillDirection.LeftToRight |
| Transform | mTrans |
| bool | mIsDirty = false |
| Camera | mCam |
| float | mOffset = 0f |
Properties | |
| float | sliderValue [get, set] |
| bool | inverted [get, set] |
| Properties inherited from UIProgressBar | |
| Transform | cachedTransform [get] |
| Cached for speed. More... | |
| Camera | cachedCamera [get] |
| Camera used to draw the scroll bar. More... | |
| UIWidget | foregroundWidget [get, set] |
| Widget used for the foreground. More... | |
| UIWidget | backgroundWidget [get, set] |
| Widget used for the background. More... | |
| FillDirection | fillDirection [get, set] |
| The scroll bar's direction. More... | |
| float | value [get, set] |
| Modifiable value for the scroll bar, 0-1 range. More... | |
| float | alpha [get, set] |
| Allows to easily change the scroll bar's alpha, affecting both the foreground and the background sprite at once. More... | |
| bool | isHorizontal [get] |
| Whether the progress bar is horizontal in nature. More... | |
| bool | isInverted [get] |
| Whether the progress bar is inverted in its behaviour. More... | |
Additional Inherited Members | |
| Public Types inherited from UIProgressBar | |
| enum | FillDirection { FillDirection.LeftToRight, FillDirection.RightToLeft, FillDirection.BottomToTop, FillDirection.TopToBottom } |
| Public Member Functions inherited from UIProgressBar | |
| delegate void | OnDragFinished () |
| virtual void | ForceUpdate () |
| Update the value of the scroll bar. More... | |
| Public Attributes inherited from UIProgressBar | |
| OnDragFinished | onDragFinished |
| Delegate triggered when the scroll bar stops being dragged. More... | |
| Transform | thumb |
| Object that acts as a thumb. More... | |
| int | numberOfSteps = 0 |
| Number of steps the slider should be divided into. More... | |
| List< EventDelegate > | onChange = new List<EventDelegate>() |
| Callbacks triggered when the scroll bar's value changes. More... | |
| Static Public Attributes inherited from UIProgressBar | |
| static UIProgressBar | current |
| Current slider. More... | |
Extended progress bar that has backwards compatibility logic and adds interaction support.
|
protected |
Position the scroll bar to be under the current touch.
|
protected |
Drag the scroll bar in the specified direction.
|
protected |
Watch for key events and adjust the value accordingly.
|
protected |
Position the scroll bar to be under the current touch.
|
protected |
Save the position of the foreground on press.
|
protectedvirtual |
Register an event listener.
Reimplemented from UIProgressBar.
|
protectedvirtual |
Upgrade from legacy functionality.
Reimplemented from UIProgressBar.
|
protected |
|
getset |
|
getset |


The Command-Line Interfaceedit
Structureedit
Curator commands can be thought of as a series of nested commands, with each stage having its own options and flags.
curator [FLAGS] COMMAND [FLAGS] SUBCOMMAND [FLAGS]
The square braces indicate optional elements. Some commands have flags, some do not. Some of those flags are optional, some are mandatory per the command. See the list of commands and flags for more information.
The first thing to know is that help is never far away. You can use the --help flag at any stage to discover which flags are available:
curator --help curator COMMAND --help curator COMMAND SUBCOMMAND --help
Understand that using the --help
flag in between two nested commands will
result in the previous level --help
output being shown.
curator --help COMMAND
will have the same output as curator --help
, and
likewise, curator COMMAND --help SUBCOMMAND
will have the same output as
curator COMMAND --help
.
The top-level help output looks like this:
$ curator --help Usage: curator [OPTIONS] COMMAND [ARGS]... Curator for Elasticsearch indices. See http://elastic.co/guide/en/elasticsearch/client/curator/current Options: --host TEXT Elasticsearch host. --url_prefix TEXT Elasticsearch http url prefix. --port INTEGER Elasticsearch port. --use_ssl Connect to Elasticsearch through SSL. --http_auth TEXT Use Basic Authentication ex: user:pass --timeout INTEGER Connection timeout in seconds. --master-only Only operate on elected master node. --dry-run Do not perform any changes. --debug Debug mode --loglevel TEXT Log level --logfile TEXT log file --logformat TEXT Log output format [default|logstash]. --version Show the version and exit. --help Show this message and exit. Commands: alias Index Aliasing allocation Index Allocation bloom Disable bloom filter cache close Close indices delete Delete indices or snapshots open Open indices optimize Optimize Indices replicas Replica Count Per-shard show Show indices or snapshots snapshot Take snapshots of indices (Backup)
Note that all available flags and commands for this level are shown. This pattern is repeated at each successive level of --help
Non-destructive Explorationedit
The show command will only ever show indices or snapshots matching the index selection or snapshot selection parameters you provide. The --dry-run flag will also show which indices or snapshots would have been acted on, but not perform the action.
With these two options, you can explore your indices and work on building a set of selection criteria without fear of making a mistake.


{% extends "analysisApp/base.html" %}
{% block js %}
{% endblock js %}
{% block titleblock %}
Home
{% endblock titleblock %}
{% block dashboard_active %}(current){% endblock dashboard_active %}
{% block content %}
Hi
{{username}},
Sentiment Analysis on last 200 tweets of {{username}}
Sentiment Analysis on last 200 mentions of {{username}}
Number of Tweets
by languages
|
Language
|
Number of Tweets
|
{% for key,value in lang_dict %}
|
{{key}}
|
{{value}}
|
{% endfor %}
Most used words in tweets
by {{username}}
|
Word
|
Number of Occurences
|
{% for key,value in cleaned_tweet_text %}
|
{{key}}
|
{{value}}
|
{% endfor %}
Top hashtags used
by {{username}}
|
Hashtag
|
Number of times used
|
{% for key,value in cleaned_hashtags %}
|
#{{key}}
|
{{value}}
|
{% endfor %}
{% endblock content %}


Tivoli Transform Bedside Speaker Into Audio Art
by Elliot NashWhat was once a staple of the bedside table has now been revived by Tivoli Audio with the release of the Revive Bluetooth bedside speaker with an inbuilt wireless charger, transforming the forgotten device into what the company calls ‘Audio Art’.
Described as being ‘so much more than a Bluetooth speaker’ the Tivoli Audio Revive bedside speaker unit will set you back $399 and is currently available for pre-order from their website.
Available in a range of finishes including white, black and a classic wood finish, the furniture grade wood cabinet enclosure of the Tivoli Revive coupled with the Danish made speaker fabric, makes for a premium-looking design.
Tivoli has chosen to omit a physical clock from the Revive design, probably due to the inclusion of the Qi wireless charger, with your smartphone or tablet taking the position of providing the time, music and news that was once provided by the bedside clock.
The Revive promises to produce high-quality audio through a simple and elegant solution with the speaker seemingly taken from the Tivoli Audio’s CUBE audio device.
Unfortunately, unlike the Tivoli Audio CUBE, the Revive is unable to be stereo paired with another speaker.
An LED Lamp has also been added to the design with five levels of brightness, adding the title of bed lamp to its list of bedroom roles.
A revival of the nightstand, the Tivoli Revive pivoting speaker head sits above the main base which holds the Qi wireless charger and a physical button array.
Unfortunately, the pivot motion is restricted to just 30 degrees of rotation (15 degrees left or right), preventing users from positioning the speaker on either side of the bed without having to reposition the entire unit.
A minor nuisance yes, but why not add a little extra rotation for the functionality improvements.
In terms of ports, the Tivoli Revive has a 3.5mm jack for direct audio input and a single 1A USB port at the back for charging another device.


Battleships
The legendary battle between two great sea captains...
We will our menu, I will create a tutorial to do that. :)
Name:
Species:
Comments:
Your comments
Are you:
Male
Female
An hermaphrodite
Asexual


Jacob
Cassell.
Portfolio
Dropdown
Hypnosis
Openbrew
Turbulant Rain
Roboclock
Links
Contact
Works are incomplete. I lost a lot of documentation and I have to go dig it up. This is just what I already had online.
Lorem Ipsum
Lorem Ipsum
Lorem Ipsum
Lorem Ipsum
Lorem Ipsum
Lorem Ipsum
Lorem Ipsum
Lorem Ipsum


Text below is piped via filter as well. But highlight doesn't appear. Class nui-ignore-highlight on container prevents it.


Tip Calculator
How much was your bill?
$
How was your service?
-- Choose an Option --
Outstanding - 30%
Good -20%
Average - 15%
Poor - 10%
Terrible - 5%
How Many People are sharing the bill?
people
Calculate!
$
0.00
each


See who we are
In the present era, the fast accumulation of data raised an acute need of information technology. We had made an attempt to step towards solution of this problem by using steganography algorith. We built StegApp(secured WebApp) is an amalgam of a complex steganalysis and a highly secured login modules to get-in to our app
198 Riverside St, Lowell, MA 01854 |<PHONE_NUMBER> | HariRamaKrishna_Gurram@student.uml.edu


Photo booth narwhal food.
Meggings literally seitan leggings.
Poke banjo hella.
Mixtape man braid YOLO lo-fi succulents.
Green juice kinfolk meggings ennui selfies.
Fixie beard raw denim blue bottle.
Edison bulb banjo brooklyn fixie pitchfork.
YOLO VHS literally, waistcoat messenger bag raw denim iPhone ugh.
Master cleanse brunch asymmetrical.
Normcore activated charcoal keytar freegan.


{% extends "general/two-column.html" %} {% set title = "Groups | MITRE ATT&CK™" %} {% set parsed = page.data | from_json %} {% set active_page = "groups" -%} {% import 'macros/navigation.html' as navigation %} {% block innerleft %}
Groups are sets of related intrusion activity that are tracked by a common name in the security community. Analysts track clusters of activities using various analytic methodologies and terms such as threat groups, activity groups, threat actors, intrusion sets, and campaigns. Some groups have multiple names associated with similar activities due to various organizations tracking similar activities by different names. Organizations' group definitions may partially overlap with groups designated by other organizations and may disagree on specific activity.
For the purposes of the Group pages, the MITRE ATT&CK team uses the term Group to refer to any of the above designations for a cluster of adversary activity. The team makes a best effort to track overlaps between names based on publicly reported associations, which are designated as “Associated Groups” on each page (formerly labeled “Aliases”), because we believe these overlaps are useful for analyst awareness. We do not represent these names as exact overlaps and encourage analysts to do additional research.
Groups are mapped to publicly reported technique use and original references are included. The information provided does not represent all possible technique use by Groups, but rather a subset that is available solely through open source reporting. Groups are also mapped to reported Software used, and technique use for that Software is tracked separately on each Software page.
| Name | Associated Groups | Description |
|---|---|---|
| {{row.name}} | {{row.aliases_list|join(", ")}} | {% if row.deprecated %} ****Deprecation Warning**** {% endif %} {{row.descr}} |


MENU
IIT Project submission
Vegetable slicer
Click to Download files
Solar dryer
Click to Download files
Palm tree climber
Click to Download files


Colors set by using color names: Note: Color names are case-insensitive. Red us tge same as red or RED.
Red
red
RED


Creating a Network¶
Having identified individuals across multiple pictures, this function generates a force directed graph based on co-appearance in images. Each individual is a node, and each co-appearance is an edge.
face_network.network(photo_dir, scale=10)
Parameters¶
- source_dir
(str) The path to the image folder
- scale
(int, default=10) Dictates the size of the nodes
Outputs¶
A file called “Image_Network.html” is created in “photo_directory/Face Network/Data/”.The graph can be opened in a web browser and is fully interactive. Hovering over a node will display a tooltip showing the cluster’s unique identifier. This corresponds to the filenames of the mosaics generated in the previous step.