One does not simply build a rich, dynamic library with a million songs. It takes an array of microservices crafted with open technologies, a database that won’t hold back performance, and an architecture that’s ready to venture out to the cloud when you are.
| With this demonstration project, you can deploy a Million Song Library and see what it’s like to launch a microservices-based application of your own. If you have questions about the demo, feel free to drop us a line at<EMAIL_ADDRESS>|
1. Million Song Library Overview
At Kenzan, we build Platform as a Service (PaaS) solutions that enable our customers to deliver secure, database-driven applications anywhere they want, anytime they need. We’re a big believer in the open source software (OSS) tools created by Netflix, along with other cool things like microservices, NoSQL databases, and scalable cloud infrastructure. These technologies, and more, help power us through all phases of delivering a customized PaaS solution, from architecture and application development to testing and DevOps.
So why did we start the Million Song Library (MSL) project? There are a couple of reasons. First, we wanted to build a model to demonstrate the capability and flexibility offered by some of the Netflix OSS components when paired with a Cassandra database. Second, we thought it would be pretty neat to have a ready-to-go foundation on top of which database-driven applications can be rapidly developed, tested, and deployed to the cloud.
And here’s the best part — we’re taking the whole thing open source. Because at Kenzan, we want everyone to have the power to build a dynamic library with a million songs. Or a million funny videos. Or a million must-have gadgets. Or a million loyal customers. Or whatever it is that you’re a big beliver in.
| Need a companion on your journey to the cloud? We’re always up for an adventure — just knock on our door at Kenzan.com. |
1.1. Key Technologies
The diagram below shows some of the key techologies we use in creating a database-driven, microservices-based Web application like the Million Song Library project. Read on for information about each of the technologies.
1.2. Microservices Architecture
In a microservices architecture, large applications are made up of multiple smaller services that can be independently deployed using automated methods. Each microservice is self-contained and provides a targeted piece of functionality. Microservices are discoverable, so clients and servers can automatically find one another. And they communicate using standard, lightweight protocols such as REST APIs.
At Kenzan, we have a lot of experience working with microservices. We’ve found that adopting a microservices architecture results in Web applications that are more modular and easier to manage compared to traditional, monolithic applications. Because microservices can be scaled independently, distributing them across servers, you have the flexibility to quickly meet changing demands. Usage of one microservice is going way up? Just add more instances of it. Need to add another cabability to your application? It’s as simple as updating a microservice or creating a new one. You don’t have to rebuild and deploy the entire application just to push out a change.
While microservices bring a lot of advantages, building a microservices-based solution from the ground up could present a challenge. That’s why the Million Song Library leverages a number of open source software (OSS) libraries from Netflix. The Netflix OSS components provide support for building applications using a suite of microservices, greatly easing the transition to a microservices-based architecture. What’s more, they’ve been tried and tested in one of the most demanding cloud environments imaginable.
Here are some of the Netflix OSS tools that make the Million Song Library possible.
- Karyon
-
Cloud infrastructure blueprint. Constructing a complex architecture requires a solid foundation. Karyon provides a scaffold for building cloud-based services, and it offers hooks for other Netflix OSS tools. When an application is based on the Karyon blueprint, it gains the ability to leverage a common core of functionality. A Karyon application can initialize itself and its dependencies, make itself available for discovery, and even let the rest of the platform know whether it’s healthy or not.
- Zuul
-
Intelligent request routing. You might remember Zuul as the gatekeeper from the movie Ghostbusters. In a microservices architecture, Zuul watches the front gate of your application and ensures that only the correct actors can access the correct services. Zuul uses pre-defined filters to allow or reject calls made to the application, and it dynamically routes requests to different application instances in response to changing demand. Zuul can even track statistics and perform stress testing to help make sure the system is prepared for times when usage spikes.
- Eureka
-
Private load balancing. Using cloud-based load balancers in a microservices architecture can quickly become cumbersome. This is due to the vast number of microservices and possible routes between them — each of which could potentially need its own load balancer. What’s more, while cloud-based load balancers are great for distributing loads for edge services that offer public access, they can’t help with middle-tier services that reside within a virtual private cloud (VPC) — services that you don’t want the outside world to acccess.
That’s where Eureka comes in. Eureka is a service registry that manages all active services and ensures that loads are properly distributed. Each time an instance of a microservice starts up, it lets Eureka know that it’s available. And if it ever stops sending out a heartbeat, Eureka quickly drops it from the list of available instances. Working in conjunction with Zuul, Eureka helps clients find active servers and gives them the information they need to communicate with one another, all within the VPC. And because server information is cached on the client, the system keeps working even if Eureka becomes unavailable.
- Archaius
-
Contextual properties management. Managing configuration properties for a large number of microservices could be problematic. Archaius solves this by storing properties in a central library where they can be easily maintained. At runtime, Archaius checks the software version and environmental values for the application — that is, the context it’s running in. Archaius determines the properties needed for that particular context, and then delivers the appropriate properties to the application over the network.
1.3. High-Performance Database
Today’s modern Web is all about data — storing it, finding it, retrieving it. And all while users are impatiently tapping their screens. As data sets grow larger, having a reliable and blazingly-fast database is critical for delivering a sucessful database-driven application. That’s why, for the Million Song Library, we went with a Cassandra database, along with a DataStax Java driver that provides an interface between the database and the application code.
Cassandra is a NoSQL database that offers much lower latency and higher scalablility than relational databases. Compared to a traditional SQL database, Cassandra can accept data at higher rates and volumes. Cassandra’s node-based architecture makes it more resilient because there is no single point of failure. And it’s easy to scale out a Cassandra database by adding more nodes (with no need to take the system down first).
Achieving such a high level of performance does require a different way of thinking. In a Cassandra database, transactions need to be simple and can’t be nested. And data may be written in multiple locations rather than normalized to a single table. But predictable application performance and constant uptime are well worth the change.
1.4. Cloud-Based Infrastructure
At Kenzan, we build a lot of PaaS solutions on Amazon Web Services (AWS). AWS provides the infrastructure and core services needed to deploy a complex, database-driven application in the cloud:
-
Elastic Compute Cloud (EC2) lets you create or decommision virtual server instances at will, so you always have just the amount of computing power you need.
-
Auto-Scaling Groups (ASGs) can fire up instances automatically in response to increased load or other conditions.
-
Amazon Machine Images (AMIs) enable a templated pipeline for successful and repeatable deployment of instances.
-
Elastic Load Balancing (ELB) distributes traffic from the public web among the available instances so that one doesn’t get overwhelmed while others sit idle.
-
Simple Storage Service (S3) provides persistant storage for AMIs, configurations, or image assets.
AWS is hosted on multiple, independent regions and availability zones that provide resiliency — even if an AWS region goes down, the application is still available. And because the middle-tier and data store reside within the Virtual Private Cloud, the entire platform is secure and protected.
| The Million Song Library architecture isn’t limited to running on AWS. It can be deployed locally. Or feel free to deploy your application to another cloud infrastructure, such as Microsoft Azure or Google Cloud Platform — whatever makes the most sense for you. |
1.5. About the MSL Demo
To take a deeper dive into the architechure of the Million Song Library project and discover more about the technologies used to build it, just keep reading — more awaits below.
When you’re ready to try out the Million Song Library demo on a Mac, Linux, or Windows computer, just follow the setup instructions, and you’ll have your own microservices-based Web application up and running in about an hour:
| If you have an AWS account, you can can also deploy the Million Song Library to an EC2 instance. See Deploying to AWS for step-by-step instructions. |
2. Code and Documentation
The Million Song Library project is open source and hosted on GitHub. The code is written using common coding standards. And it’s fully documented with the help of automated documentation tools.
2.1. GitHub Repositories
You can find all of the source code for the Million Song Library project in the master repository on GitHub.
The master repository includes multiple sub-repositories that have been added as Git submodules using the git submodule add
command. There is one submodule for each service, configuration, and UI service. Dividing the project into submodules like this makes it easier to configure continuous integration (CI) jobs. It also provides a cleaner separation of responsibilities within the code. At the same time, having one master repository means that it’s simple to retrieve the entire project all at once.
2.2. Coding Standards
We think the Google coding standards and style guides are concise and useful, so we based our own style guide on them. We wrote all of the code in the Million Song Library project to conform to these standards. See the Kenzan Style Guide to learn more.
2.3. Documentation Tools
The Million Song Library project leverages a number of tools to generate API, code, and other documentation.
2.3.1. Swagger API Documentation and Code Generation
We use the open-source Swagger framework to create the API documentation right alongside the client and server code. Swagger provides more than just a way to document as you develop because the documentation actually drives code generation. Using the Swagger codegen tools, you can generate client libraries and server stubs from the same definition file used to generate the documentation. By unifying the generation of the API documentation with the application code, Swagger assures that they always stay in sync. And it enables publishing of the API documentation to both static and interactive formats.
| Recently, Swagger was donated to the Open API Initiative and was renamed the OpenAPI Specification. |
To use Swagger, we first specify the API methods and models in multiple YAML files and combine them as necessary:
-
definitions.yaml - (One file) Declares all of the objects that will be passed into and out of the APIs.
-
paths.yaml - (One file for each microservice) Declares the API methods for that microservice.
-
index.yaml - (One file for each microservice) Uses
$ref
commands to combine certain other YAML files to completely define the API for that microservice. -
index.yaml - (One file) Combines all of the API methods and models into a single file that is used to generate documentation for all API methods in all microservices.
Next, to generate documentation and server-side code stubs, we use JavaScript (resolve.js) to resolve the $ref
commands and output the final YAML files. The final YAML files are sent through Swagger (swagger-client-cli) to build the server-side code stubs and the HTML API documentation.
| For an example of static documentation generated with Swagger, see this static HTML representation. For instructions on how to run the Swagger mock server and view the interactive documentation, see the “Swagger” section in the README for the msl-pages sub-repository. |
2.3.2. Javadoc and ESDoc
To generate documentation for the server Java classes, we use the open-source Javadoc tool. Information about each class, including its parameters and return object or value, is added to the code by means of standard comments. HTML or PDF documentation can then be generated from the code comments using command line tools. A number of popular IDEs also offer tools for generating JavaDocs.
Similarly, we use ESDoc to create code documentation for the JavaScript functions and methods that are part of the UI service.
2.3.3. AsciiDoc Documentation
We use AsciiDoc to create supplemental documentation for the Million Song Library project (including the document you’re reading now).
AsciiDoc is a text-based markup language for writing documentation such as articles, books, and more. Like other lightweight markup languages, AsciiDoc has a simple syntax that is human readable. Using open source tools, text files written in AsciiDoc can be easily processed to create output in a variety of formats, including HTML and PDF.
3. Client/UI Architecture
The Million Song Library client architecture is modular by design and takes advantage of modern Web application languages and frameworks.
3.1. AngularJS
The Million Song Library UI presents itself to the user as a single-page app built using AngularJS. AngularJS is an open-source JavaScript framework that extends HTML and enables the creation of dynamic Web applications. It provides a flexible toolset that works well with other libraries.
3.2. Sass
We use Sass (along with SCSS syntax) to help manage the complex CSS stylesheets that make the Million Song Library UI possible. Sass is a CSS pre-processor that extends the CSS language. It adds features that allow the use of variables, mixins, functions, and other techniques that make CSS much more maintainable, themable, and extendable.
3.3. Material Design
Underlying the CSS and Angular architecture of the Million Song Library UI is Google’s Material Design, which provides an accessible look and feel that scales across browsers and devices. We use the Angular Material version of Material Design to leverage its built-in AngularJS components. Adopting Material Design allows for extensibility of the Million Song Library UI and makes it easy for others to add their own features.
3.4. KSS Styleguide Generation
KSS provides a specification and styleguide format that helps you produce CSS documentation as well as example HTML that shows your styles in action. The KSS documentation syntax is readable by humans, yet is structured enough to be automatically parsed by a machine. It’s designed with CSS preprocessors (such as Sass or Less) in mind, and is flexible enough to accommodate a multitude of CSS frameworks (such as YUI, Blueprint, or 960).
3.5. ES2015
The Million Song Library UI uses new features (like await
and async
) available in the latest version of the JavaScript programming language, ES2015. We use Babel to transpile ES2015 script into standard JavaScript syntax that today’s browsers can understand.
3.6. Webpack
Webpack is a module bundler. We use it to produce static assets from modules that have dependencies. The main index.html
file is built using the HTML Webpack Plugin.
4. Server Architecture
Instead of a traditional edge/middle architecture, the Million Song Library project uses a simplified edge/data client architecture.
4.1. Data Client Layer
The data clients are JARs, each one containing the methods and data transfer objects (DTOs) needed to access all of the tables in a Cassandra cluster.
To enhance scalability and configuration flexibility, the Cassandra tables are split into three independent clusters: account, catalog, and rating. Each of these clusters has a data client JAR dedicated to accessing it: account-data-client, catalog-data-client, and rating-data-client, respectively. This means that a microservice that needs to access Cassandra data will include one or more of the data client JARs.
4.1.1. Data Client DTOs and Methods
The tables below list the DTOs and methods available for each database table accessed by account-data-client, catalog-data-client, and rating-data-client.
| For details about all of the available DTOs and methods, see the Javadoc documentation for each data client. |
| Table | DTO |
|---|---|
users |
UserDto |
songs_by_user |
SongsByUserDto |
albums_by_user |
AlbumsByUserDto |
artists_by_user |
ArtistsByUserDto |
| Table | Method |
|---|---|
users |
Observable<Void> addOrUpdateUser(UserDto) |
Observable<UserDto> getUser(UUID userUuid) |
|
Observable<Void> deleteUser(UUID userUuid) |
|
songs_by_user |
Observable<Void> addOrUpdateSongsByUser(SongsByUserDto) |
Observable<SongsByUserDto> getSongsByUser(UUID userId, Timestamp favoritesTimestamp, UUID songUuid) |
|
Observable<ResultSet> getSongsByUser(UUID userId, Timestamp favoritesTimestamp, Optional<Integer> limit) |
|
Observable<ResultSet> getSongsByUser(UUID userId, Optional<Integer> limit) |
|
Observable<Result<SongsByUserDto> map(Observable<ResultSet>) |
|
Observable<Void> deleteSongsByUser(UUID userId, Timestamp favoritesTimestamp, UUID songUuid) |
|
albums_by_user |
Observable<Void> addOrUpdateAlbumsByUser(AlbumsByUserDto) |
Observable<AlbumsByUserDto> getAlbumsByUser(UUID userId, Timestamp favoritesTimestamp, UUID albumUuid) |
|
Observable<ResultSet> getAlbumsByUser(UUID userId, Timestamp favoritesTimestamp, Optional<Integer> limit) |
|
Observable<ResultSet> getAlbumsByUser(UUID userId, Optional<Integer> limit) |
|
Observable<Result<AlbumsByUserDto> map(Observable<ResultSet>) |
|
Observable<Void> deleteAlbumsByUser(UUID userId, Timestamp favoritesTimestamp, UUID albumUuid) |
|
artists_by_user |
Observable<Void> addOrUpdateArtistsByUser(ArtistsByUserDto) |
Observable<ArtistsByUserDto> getArtistsByUser(UUID userId, Timestamp favoritesTimestamp, UUID artistUuid) |
|
Observable<ResultSet> getArtistsByUser(UUID userId, Timestamp favoritesTimestamp, Optional<Integer> limit) |
|
Observable<ResultSet> getArtistsByUser(UUID userId, Optional<Integer> limit) |
|
Observable<Result<ArtistsByUserDto> map(Observable<ResultSet>) |
|
Observable<Void> deleteArtistsByUser(UUID userId, Timestamp favoritesTimestamp, UUID artistUuid) |
| Table | DTO |
|---|---|
featured_songs |
FeaturedSongsDto |
songs_by_facet |
SongsByFacetDto |
featured_albums |
FeaturedAlbumsDto |
albums_by_facet |
AlbumsByFacetDto |
featured_artists |
FeaturedArtistsDto |
artists_by_facet |
ArtistsByFacetDto |
songs_albums_by_artist |
SongsAlbumsByArtistDto |
songs_artist_by_album |
SongsArtistByAlbumDto |
album_artist_by_song |
AlbumArtistBySongDto |
paging_state |
PagingStateDto |
| Table | Method |
|---|---|
featured_songs |
Observable<ResultSet> getFeaturedSongs(Optional<Integer> limit) |
Observable<Result<FeaturedSongsDto> map(Observable<ResultSet>) |
|
songs_by_facet |
Observable<ResultSet> getSongsByFacet(String facetName, Optional<Integer> limit) |
Observable<Result<SongsByFacetDto> map(Observable<ResultSet>) |
|
featured_albums |
Observable<ResultSet> getFeaturedAlbums(Optional<Integer> limit) |
Observable<Result<FeaturedAlbumsDto> map(Observable<ResultSet>) |
|
albums_by_facet |
Observable<ResultSet> getAlbumsByFacet(String facetName, Optional<Integer> limit) |
Observable<Result<AlbumsByFacetDto> map(Observable<ResultSet>) |
|
featured_artists |
Observable<ResultSet> getFeaturedArtists(Optional<Integer> limit) |
Observable<Result<FeaturedArtistsDto> map(Observable<ResultSet>) |
|
artists_by_facet |
Observable<ResultSet> getArtistsByFacet(String facetName, Optional<Integer> limit) |
Observable<Result<ArtistsByFacetDto> map(Observable<ResultSet>) |
|
songs_albums_by_artist |
Observable<ResultSet> getSongsAlbumsByArtist(UUID artistUuid, Optional<Integer> limit) |
Observable<Result<SongsAlbumsByArtistDto> map(Observable<ResultSet>) |
|
songs_artist_by_album |
Observable<ResultSet> getSongsArtistByAlbum(UUID albumUuid, Optional<Integer> limit) |
Observable<Result<SongsArtistByAlbumDto> map(Observable<ResultSet>) |
|
album_artist_by_song |
Observable<ResultSet> getAlbumArtistBySong(UUID songUuid, Optional<Integer> limit) |
Observable<Result<AlbumArtistBySongDto> map(Observable<ResultSet>) |
|
paging_state |
Observable<Void> addOrUpdatePagingState(PagingStateDto) |
Observable<PagingStateDto> getPagingState(UUID pagingStateUuid) |
|
Observable<Void> deletePagingState(UUID pagingStateUuid) |
| Table | DTO |
|---|---|
average_ratings |
AverageRatingDto |
user_ratings |
UserRatingsDto |
| Table | Method |
|---|---|
average_ratings |
Observable<Void> addOrUpdateAverageRating(AverageRatingDto) |
Observable<AverageRatingDto> getAverageRating(UUID contentId, String contentType) |
|
Observable<Void> deleteAverageRating(UUID contentId, String contentType) |
|
user_ratings |
Observable<Void> addOrUpdateUserRatings(UserRatingsDto) |
Observable<UserRatingsDto> getUserRatings(UUID userUuid, String contentType, UUID contentUuid) |
|
Observable<ResultSet> getUserRatings(UUID userUuid, String contentType, Optional<Integer> limit) |
|
Observable<ResultSet> getUserRatings(UUID userUuid, Optional<Integer> limit) |
|
Observable<Result<UserRatingsDto> map(Observable<ResultSet>) |
|
Observable<Void> deleteUserRatings(UUID userUuid, String contentType, UUID contentUuid) |
4.2. APIs
To learn more about the Million Song Library server APIs, see the API documentation we generated using Swagger. You can find links to all of the documentation in the README file on the Million Song Library GitHub page.
4.3. Data POJOs
The server has three different types of data POJOs (plain old Java objects):
-
Models - Classes that represent the data payloads that are passed between client and server.
-
Data Transfer Objects (DTOs) - Classes that represent the data persisted in Cassandra.
-
Business Objects (BOs) - Classes that help to convert data between models and DTOs.
4.3.1. Models
The model classes are generated by Swagger and represent the data payloads that are passed back and forth between client and server.
4.3.2. Data Transfer Objects (DTOs)
Data Transfer Objects represent the data as it is persisted in Cassadra. Simply, there is a one-to-one relationship between Cassandra tables and concrete DTO classes. In many cases this data is optimized for Cassandra, which means it needs to undergo some manipulation to convert it to a model.
4.3.3. Business Objects (BOs)
Insider the server, business object classes represent the glue that holds together model classes and DTO classes. In many cases there is not a simple one-to-one connection between reading a DTO from Cassandra and sending that data as a model to the client. For example, when building an AlbumInfo model, the album’s metadata, community, and user ratings need to be merged. The BO contains the merged representation of the data.
4.4. Languages and Frameworks
The back-end server is written in Java — Java Enterprise Edition 1.8, to be exact. Communication between client and server is performed using RESTful Web services over HTTP. The REST code stubs are generated by Swagger and use JAX-RS annotations. Java’s Jersey library is used as the implementation of the JAX-RS API. Finally, JUnit is used as our unit testing framework.
As described in the Microservices Architecture section above, portions of the Netflix OSS stack are used to provide cloud services. At its foundation, the server is based on Karyon, which enables the use of other Netflix OSS tools. For example, Archaius is used as the configuration manager, allowing a service to dynamically react to changes in configuration parameters.
All data is persisted using Apache’s Cassandra. Cassandra is a scalable, highly-available NoSQL database. The DataStax Java driver and row→POJO mapper provide an interface between the Java code and the Cassandra database.
The table below summarizes the languages and frameworks used in the Million Song Library server.
| Use | Name | Vendor | Link |
|---|---|---|---|
Language |
Java EE 1.8 |
Oracle |
|
API Documention/Code Generator |
Swagger |
open-souce |
|
ReST API |
JAX-RS |
Oracle |
|
ReST Implementation |
Jersey |
Oracle |
|
Unit Testing |
JUnit |
JUnit |
|
Cloud Infrastructure |
Karyon |
Netflix OSS |
|
Configuration Management |
Archaius |
Netflix OSS |
|
Database |
Cassandra |
Apache |
|
Database Driver |
DataStax |
DataStax |
|
Database Row→POJO Mapper |
Datastax |
DataStax |
|
Reactive Code Library |
RxJava |
Netflix |
4.5. Album Cover Artwork
Album cover artwork is retrieved from MusicBrainz, “an open music encyclopedia that collects music metadata and makes it available to the public”, and Cover Art Archive, “whose goal is to make cover art images available to everyone on the Internet in an organised and convenient way”. The Million Song Library data set contains an artist’s MusicBrainz ID (artist_mbid). As part of the data import process, this identifier is used to retrieve the links to artwork using the MusicBrainz and Cover Art Archive APIs.
| All interactions with MusicBrainz and the Cover Art Archive are performed during the data import process — no queries to either of these sites occur when the Million Song Library server is running. |
4.5.1. MusicBrainz Access
We use the musicbrainzws2-java Java library to access the MusicBrainz API.
4.5.2. Parsing and Using MusicBrainz Data
Multiple releases (that is, albums) can be returned for the requested artist. The data import code reviews the available releases/albums for the artist and selects album art using the following algorithm:
-
Consider only those releases/albums whose title matches (not case-sensitive) the name of the album from the Million Song Library data.
-
Consider only those release/albums for which
cover-art-archive/front = true
. -
Give preference to multiple possible candidates based on packaging in this order:
-
Jewel Case (indicates a CD)
-
Cardboard/Paper Sleeve (idicates an LP)
-
Cassette Case (indicates cassette tape)
-
Other packaging
-
If a particular piece of art is not available, the image link field in the database is left blank/null. If artwork is available, then the data import process retrieves the image URL through the Cover Art Archive REST API (using CoverArtArchiveClient) and writes it to the database.
4.5.3. Cover Art Archive Access
The CoverArtArchiveClient class from the Cover Art Archive API allows us to retrieve the image URL using the release MBID (MusicBrainz Identifier).
4.5.4. Parsing and Using CoverArtArchive Data
As you can see in the code example above, multiple images can be returned for the requested release/album. The data import code will review the available images for the release/album and select the image URL using the following algorithm:
-
Consider only those images for which
isFront() == true
. -
Give preference to multiple possible URLs in this order:
-
small thumbnail
-
large thumbnail
-
image (this is the high resolution image)
-
If, based on this algorithm, a particular piece of art is not available, the image link field in the database is left blank/null.
4.5.5. “Image Unavailable” Images
When preparing to send an artwork link in response to a request, if the image link URL from the database is blank/null, the server instead inserts a URL that points to an “artwork unavailable” image on the pages server. This URL is defined as a configuration parameter. Three “artwork unavailable” images (one each for album, artist, and song) are available for HTTP retrieval from the pages server.
5. Database Architecture
Due to the large amount of data to be persisted, and the need for quick reads and linear scalable performance, the Million Song Library project uses Cassandra as the back-end datastore, along with the Datastax Java driver as the interface.
5.1. Design
We used the Kashlev Data Modeler (KDM) to model the Cassandra schema.
5.1.1. Entity-Relationship Model
Initially, we performed a traditional data analysis — looking at what entities are required based on the use cases, and what the relationships are between those entities. The result is illustrated in the entity-relationship model below which was created with KDM and uses Chen’s notation.
5.1.2. Access Patterns
Next, we used those same use cases to determine how the data in the entities would be accessed. Given the use patterns of a Cassandra datastore, it’s important to think in terms of queries when designing the schema. This can be a challenging transition for those accustomed to building SQL schemas. Maybe even more challenging is the need to denormalize the schema — a thought that could keep an SQL database designer up at night. Luckily, there are many good tutorials and documents online dicussing this process.
5.2. Implementation
The following table shows the details for each query included in the access patterns diagram above.
| Fields marked with K are partition keys. Fields marked with C are cluster keys. |
| Query Number | Description | Table Schema |
|---|---|---|
Q1 |
Users |
|
Q2 |
User ratings |
|
Q3 |
Average Ratings |
|
Q4 |
Featured Songs |
|
Q5 |
Songs by Facet |
|
Q6 |
Featured Songs |
|
Q7 |
Songs by Facet |
|
Q8 |
Featured Artists |
|
Q9 |
Artists by Facet |
|
Q10 |
Songs/Albums by Artist |
|
Q11 |
Songs by User |
|
Q12 |
Albums by User |
|
Q13 |
Artists by User |
|
Q14 |
Songs/Artist by Album |
|
Q15 |
Album/Artist by Song |
|
Q16 |
Paging State |
5.3. Data Import
The Million Song Library project uses data from the Million Song Dataset. (A 10,000 song subset of the full data set is also available.) Downloading the data set yields a gzipped file that contains two directories: AdditionalFiles and data. The data directory has a number of subdirectories that contain an HDF file (with the extension h5) for each song in the data set. The AdditionalFiles directory contains several files that summarize the data set — these files are not used for the import.
HDF or Hierarchical Data Format files contain all the data needed for albums, artists, and songs. We use the Python library written by Thierry Bertin-Mahieux to read these files. (We initially tried using Java and JHI5 but ran into issues with the native library aspect of that implementation.)
At a high level, the data import process uses Python to traverse a given directory tree, consuming all of the .h5 files and creating a single CSV file with one line for each song. This CSV file is then consumed by a Java program to create the CSVs that are inserted into the Cassandra cluster.
| The tags within an .h5 file (/metadata/artist_terms) essentially indicate the genre of the song. There are so many genres used in the data set that it makes them not particularly useful as a searching mechanism. For this reason, a list of genres is defined in Genre.java to declare which of the genres we are interested in persisting. Any genres not found in the list are ignored. |
6. Mac OS X Automated Setup
Follow the steps in this section to set up the Million Song Library demo on a Mac.
6.1.
- Mac OS X 10.11
-
We tested this procedure on Mac OS X 10.11 (El Capitan). Check your OS X version to see if you’re already running El Capitan. If you’re running an older version of OS X, you should upgrade your Mac to El Capitan.
- Apple Xcode
-
On Mac OS X, some necessary command line tools aren’t installed by default, so to get them you’ll need to install the latest version of Xcode from the Mac App Store.
After installation, make sure to open Xcode, accept the license agreement, and (if prompted) install any additional tools. - Java 1.8 JDK
-
Install the Java 1.8 JDK. Make sure to select the Mac OS X x64 download option. To install, just double-click the DMG file to open it, and then double-click the installer.
- Git
-
Install Git if you don’t have it already. One easy method is to download the Mac OS X installer. To install, just double-click the DMG file to open it, and then double-click the installer.
- Terminal
-
You’ll need a terminal application so you can enter commands at the command prompt. The Terminal app included with Mac OS X will work just fine. You can find it in Applications→Utilities.
Where is My Home Directory?
For this process, it’s important to know where your home directory is. Your home directory is the directory that has the same name as your user name.
For example, if the user name you use to log on to your Mac is jdoe, your home directory is named jdoe. In Terminal, your home directory is represented by the tilde (~) sign. For example, the path ~/stuff points to a directory named stuff located inside your home directory.
6.2. Update Existing Tools (If Needed)
The Million Song Library setup script will automatically install the required tools and frameworks, so in most cases there’s nothing more you need to install before getting started. There’s one exception—if you already have some of the required tools installed on your system, then you need to make sure they are at the required version. Check the list below.
- Maven 3.3.9 or later
-
To see what version of Maven is installed, enter
mvn -version
at the command prompt. If you get the messagecommand not found
, it means Maven is not installed, and there’s no need to do anything.If the command reports that an older version is installed, upgrade your Maven installation to version 3.3.9 or later.
- Node 0.12.x or later
-
To see what version of Node is installed, enter
node --version
at the command prompt. If you get the messagecommand not found
, it means Node is not installed, and there’s no need to do anything.If the command reports that an older version is installed, download the Node.js Mac OS X installer. Just double-click the installer to run it. This will also install npm.
- npm 2.7.x or later
-
To see what version of npm is installed, enter
npm -version
at the command prompt. If you get the messagecommand not found
, it means npm is not installed, and there’s no need to do anything.If the command reports that an older version is installed, download the Node.js Mac OS X installer, which includes npm. Just double-click the installer to run it.
6.3. Enable the Root User
The setup script requires the use of sudo
to run certain commands as root. However, on Mac OS X, the root user is not enabled by default. You’ll need to enable the root user so that the setup script will run properly.
To enable the root user, just follow these instructions.
6.4. Clone the MSL Repository
To run the setup script, you first need to clone the million-song-library repository to a local directory.
-
Open a new Terminal window and make sure you are in your home directory.
-
Enter the following command:
git clone https://github.com/kenzanmedia/million-song-library
Git creates a directory called million-song-library inside your home directory, and checks out a working copy of the repository.
-
Enter the following command to change directories to the cloned repository:
cd million-song-library
-
Enter the folowing command to initialize the repository and submodules:
git submodule init && git submodule update
6.5. Install and Start Cassandra
Now let’s get Cassandra up and running.
-
Download Cassandra 2.1.11 (Community version) from the Datastax download site.
Make sure to download the correct file: dsc-cassandra-2.1.11-bin.tar.gz -
Create a new directory named cassandra inside your home ( ~/ ) directory.
-
Move the dsc-cassandra-2.1.11-bin.tar.gz file into the ~/cassandra directory, and then double-click the file to extract it.
-
To start Cassandra, open a new Terminal window and enter the following command:
sh ~/cassandra/dsc-cassandra-2.1.11/bin/cassandra
| If you get an error when trying to start Cassandra, it probably means that Cassandra is already running (for example, from running the Million Song Library demo previously). To correct this issue, reboot your system, or stop Cassandra (see Stop the MSL Demo). Then try starting Cassandra again. You should always make sure to stop Cassandra when you’re done working with the Million Song Library demo. |
6.6. Run the Setup Script
Now that Cassandra is running, you’re ready to run the setup script.
| If you’ve already run the setup script once, you don’t need to run it again to start the demo. Just skip ahead to Start the MSL Demo. If you haven’t run the setup script before, follow the steps below. |
-
In Terminal, press Command+T to open a new tab.
-
Enter the following command to change to the script directory:
cd ~/million-song-library/bin
-
Enter the following command to make the setup script executable:
chmod +x setup.sh
-
Enter the following command to run the setup script:
./setup.sh -c ~/cassandra/dsc-cassandra-2.1.11 -n -s -g
Watch the script output — you will be prompted to press <Enter> and also to enter your administrator password. -
Wait for the script to complete (about 30 minutes).
| It’s normal to see some warnings while the script runs. If the script encounters an error that prevents it from finishing, make sure you installed all the required tools. See Get Your System Ready and Update Existing Tools (If Needed) above. Then try running the script again. |
6.7. Start the MSL Demo
Setup is complete — now it’s time to start the Million Song Library demo.
-
In Terminal, press Command+T to open another new tab.
-
Enter the following command to change to the application directory:
cd ~/million-song-library/msl-pages
-
Enter the following command to start the application front end:
npm run full-dev
Wait for the front end to start up — this will take just a couple of minutes.
-
In Terminal, press Command+T to open another new tab. You should still be in the msl-pages directory.
Make sure to open a new Terminal tab or window before starting the servers, to avoid stopping the front end. -
Enter the following command to start the server instances:
npm run serve-all
Wait for the server instances to start up — again, this will take just a couple of minutes.
-
Open a Web browser and point it to: msl.kenzanlabs.com:3000
The Million Song Library home page displays.
If you don’t see data right away, wait a few minutes and then refresh the page. If data still doesn’t load, try clearing your web browser cache. Million Song Library Home Page
Now that the Million Song Library demo is working, here are some fun things to try:
-
Click Register to register for an account.
-
Click the labels to switch the view between Songs, Albums, and Artists.
-
Click a genre or star rating on the left to filter songs. (Click the x to clear a filter.)
-
Mouse over a song and click Add to library to add it to your music library (you must be logged in).
-
Click the stars below a song to rate it (you must be logged in).
6.8. Stop the MSL Demo
Done having fun for now? Follow these steps to stop the Million Song Library demo.
-
In Terminal, switch to the tab where you started the server instances and press Control+C to stop the server.
-
In Terminal, switch to the tab where you started the application and press Control+C to stop the application.
-
In Terminal, switch to the tab where you started Cassandra and enter the following command:
ps auwx | grep cassandra
Look at the output from the command and note the first 3–5 digit number that appears in the output. This is the process ID for Cassandra.
-
Enter the following command where pid is the process ID you found (you’ll be prompted for your administrator password):
sudo kill pid
| You don’t need to run the setup script if you want to start the Million Song Library demo again. Just start Cassandra (see Install and Start Cassandra) and then start the application and server instances (see Start the MSL Demo). |
7. Linux Automated Setup
Follow the steps in this section to set up the Million Song Library demo on a system running Linux.
7.1.
- Ubuntu 14.04
-
We tested this procedure on Ubuntu 14.04 (Trusty Tahr). Check your Ubuntu version to see if you’re already running Trusty Tahr. If you’re running an older version of Ubuntu, you should upgrade your system to Trusty Tahr.
- Java 1.8 JDK
-
Install the Java 1.8 JDK:
-
In a Terminal window, enter the following command to add the WebUpd8 Java PPA repository (you’ll be prompted for your administrator password):
sudo add-apt-repository ppa:webupd8team/java
-
Enter the following command to update the package index:
sudo apt-get update
-
Enter the following command to install the Java 1.8 JDK:
sudo apt-get install oracle-java8-installer
-
- Git
-
Install Git if you don’t have it already. In a Terminal window, enter the following command (you’ll be prompted for your administrator password):
sudo apt-get install git
- Terminal
-
You’ll need a terminal application so you can enter commands at the command prompt. The Terminal app included with Ubuntu will work just fine. You can find it in Applications.
Where is My Home Directory?
For this process, it’s important to know where your home directory is. Your home directory is the directory that has the same name as your user name.
For example, if the user name you use to log on to your system is jdoe, your home directory is named jdoe. In Terminal, your home directory is represented by the tilde (~) sign. For example, the path ~/stuff points to a directory named stuff located inside your home directory.
7.2. Update Existing Tools (If Needed)
The Million Song Library setup script will automatically install the required tools and frameworks, so in most cases there’s nothing more you need to install before getting started. There’s one exception—if you already have some of the required tools installed on your system, then you need to make sure they are at the required version. Check the list below.
- Maven 3.3.9 or later
-
To see what version of Maven is installed, enter
mvn -version
at the command prompt. If the system does not return a version number, it means Maven is not installed, and there’s no need to do anything.If the command reports that an older version is installed, upgrade your Maven installation to version 3.3.9 or later.
- Node 0.12.x or later
-
To see what version of Node is installed, enter
node --version
at the command prompt. If the system does not return a version number, it means Node is not installed, and there’s no need to do anything.If the command reports that an older version is installed, update your installation of Node.js.
- npm 2.7.x or later
-
To see what version of npm is installed, enter
npm -version
at the command prompt. If the system does not return a version number, it means npm is not installed, and there’s no need to do anything.If the command reports that an older version is installed, update your installation of Node.js, which includes npm.
7.3. Clone the MSL Repository
To run the setup script, you first need to clone the million-song-library repository to a local directory.
-
Open a new Terminal window and make sure you are in your home directory.
-
Enter the following command:
git clone https://github.com/kenzanmedia/million-song-library
Git creates a directory called million-song-library inside your home directory, and checks out a working copy of the repository.
-
Enter the following command to change directories to the cloned repository:
cd million-song-library
-
Enter the folowing command to initialize the repository and submodules:
git submodule init && git submodule update
7.4. Install and Start Cassandra
Now let’s get Cassandra up and running.
-
Download Cassandra 2.1.11 (Community version) from the Datastax download site.
Make sure to download the correct file: dsc-cassandra-2.1.11-bin.tar.gz -
Create a new directory named cassandra inside your home ( ~/ ) directory.
-
Move the dsc-cassandra-2.1.11-bin.tar.gz file into the ~/cassandra directory, and then extract the contents into the ~/cassandra directory.
-
To start Cassandra, enter the following command in Terminal:
sh ~/cassandra/dsc-cassandra-2.1.11/bin/cassandra
| If you get an error when trying to start Cassandra, it probably means that Cassandra is already running (for example, from running the Million Song Library demo previously). To correct this issue, reboot your system, or stop Cassandra (see Stop the MSL Demo). Then try starting Cassandra again. You should always make sure to stop Cassandra when you’re done working with the Million Song Library demo. |
7.5. Run the Setup Script
Now that Cassandra is running, you’re ready to run the setup script.
| If you’ve already run the setup script once, you don’t need to run it again to start the demo. Just skip ahead to Start the MSL Demo. If you haven’t run the setup script before, follow the steps below. |
-
In Terminal, press Shift+Control+T to open a new tab.
-
Enter the following command to change to the script directory:
cd ~/million-song-library/bin
-
Enter the following command to make the setup script executable:
chmod +x setup.sh
-
Enter the following command to run the setup script (you’ll be prompted for your administrator password):
sudo ./setup.sh -c ~/cassandra/dsc-cassandra-2.1.11 -n -s -g
-
Wait for the script to complete (about 30 minutes).
| It’s normal to see some warnings while the script runs. If the script encounters an error that prevents it from finishing, make sure you installed all the required tools. See Get Your System Ready and Update Existing Tools (If Needed) above. Then try running the script again. |
7.6. Start the MSL Demo
Setup is complete — now it’s time to start the Million Song Library demo.
-
In Terminal, press Shift+Control+T to open another new tab.
-
Enter the following command to change to the application directory:
cd ~/million-song-library/msl-pages
-
Enter the following command to start the application front end:
npm run full-dev
Wait for the front end to start up — this will take just a couple of minutes.
-
In Terminal, press Shift+Control+T to open another new tab. You should still be in the msl-pages directory.
Make sure to open a new Terminal tab or window before starting the servers, to avoid stopping the front end. -
Enter the following command to start the server instances (you’ll be prompted for your administrator password):
sudo npm run serve-all
Wait for the server instances to start up — again, this will take just a couple of minutes.
-
Open a Web browser and point it to: msl.kenzanlabs.com:3000
The Million Song Library home page displays.
If you don’t see data right away, wait a few minutes and then refresh the page. If data still doesn’t load, try clearing your web browser cache. Million Song Library Home Page
Now that the Million Song Library demo is working, here are some fun things to try:
-
Click Register to register for an account.
-
Click the labels to switch the view between Songs, Albums, and Artists.
-
Click a genre or star rating on the left to filter songs. (Click the x to clear a filter.)
-
Mouse over a song and click Add to library to add it to your music library (you must be logged in).
-
Click the stars below a song to rate it (you must be logged in).
7.7. Stop the MSL Demo
Done having fun for now? Follow these steps to stop the Million Song Library demo.
-
In Terminal, switch to the tab where you started the server instances and press Control+C to stop the server.
-
In Terminal, switch to the tab where you started the application and press Control+C to stop the application.
-
In Terminal, switch to the tab where you started Cassandra and enter the following command:
ps auwx | grep cassandra
Look at the output from the command and note the first 3–5 digit number that appears in the output. This is the process ID for Cassandra.
-
Enter the following command where pid is the process ID you found (you’ll be prompted for your administrator password):
sudo kill pid
| You don’t need to run the setup script if you want to start the Million Song Library demo again. Just start Cassandra (see Install and Start Cassandra) and then start the application and server instances (see Start the MSL Demo). |
8. Windows Automated Setup
Follow the steps in this section to set up the Million Song Library demo on a system running Windows. You will actually be installing the demo inside a Linux virtual machine using Vagrant — a process that’s both easy and free.
8.1.
- Windows 10
-
We tested this procedure on Windows 10 Pro. Check your Windows version to see if you’re already running Windows 10 Pro, and to find out if you’re running a 32-bit or 64-bit version of Windows. If you’re running an older version of Windows, you should upgrade your system to Windows 10.
- Git
-
Install Git if you don’t have it already. One easy method is to download the Windows installer. To install, just double-click the EXE file.
During Git installation, make sure to select the Use Git from the Windows Command Prompt option. Otherwise you won’t be able use Git commands in PowerShell. Installing Git for Windows - Vagrant
-
The easiest way to set up the Million Song Library on Windows is with Vagrant, a tool for automating the creation of development environments. If you don’t already have it, install Vagrant.
- PowerShell
-
You’ll need to use Windows PowerShell to enter commands at the command prompt. You can find it in Start→All Apps→Windows PowerShell.
For some commands, you’ll need to use Git Bash instead (we’ll let you know when). You can find it in Start→All Apps→Git→Git Bash.
Where is My Home Directory?
For this process, it’s important to know where your home directory is. Your home directory is the directory that has the same name as your user name.
For example, if the user name you use to log on to Windows is jdoe, then the path C:\Users\jdoe points to your home directory.
In Git Bash, your home directory is represented by the tilde (~) sign. For example, the path ~/stuff points to a directory named stuff located inside your home directory.
8.2. Clone the MSL Repository
To use Vagrant to set up a virtual machine for the demo, you first need to clone the million-song-library repository to a local directory.
-
Open a new PowerShell window and make sure you are in your home directory.
-
Enter the following command:
git clone https://github.com/kenzanmedia/million-song-library
Git creates a directory called million-song-library inside your home directory, and checks out a working copy of the repository.
-
Enter the following command to change directories to the cloned repository:
cd million-song-library
-
Enter the folowing command to initialize the repository:
git submodule init
-
Enter the folowing command to update all submodules:
git submodule update
8.3. Set up the Virtual Machine
The Vagrant file contains all of the information needed to set up and configure the Linux virtual machine to run the Million Song Library Demo. Let’s get it running.
-
Open a new PowerShell window and make sure you are in your home directory.
-
Enter the following command to change to the million-song-library repository:
cd million-song-library
-
Enter the following command to set up the virtual machine:
vagrant up ubuntu
-
Wait for the setup process to complete (about 30 minutes).
If you get an error stating The box 'ubuntu/trusty64' could not be found , it means you don’t have the correct version of the Microsoft Visual C++ Redistributable Package on your system. First install the Microsoft Visual C++ 2010 SP1 Redistributable Package, and then run the vagrant up ubuntu command again.
|
8.4. Edit the Hosts File
While you’re waiting for the setup process to complete, take a few minutes to update the hosts file on your local system. This will allow you to access the Million Song Library application using a Web browser.
-
In a text editor, open the hosts file located in the C:\Windows\System32\Drivers\etc directory.
-
Add the following line to the hosts file.
<IP_ADDRESS> msl.kenzanlabs.com
-
Save and close the hosts file.
8.5. Start the MSL Demo
This is where we need to start using Git Bash, which already has an ssh client (unlike PowerShell).
-
Open a new Git Bash window and make sure you are in your home directory.
-
Enter the following command to change to the million-song-library repository:
cd million-song-library
-
Enter the following command to connect to the virtual machine and forward the necessary ports (it’s really long, so use copy and paste, and make sure to select the entire command):
vagrant ssh ubuntu -- -L 3000:localhost:3000 -L 3002:localhost:3002 -L 3003:localhost:3003 -L 3004:localhost:3004 -L 9002:localhost:9002 -L 9003:localhost:9003
-
After you are connected to the virtual machine, enter the following command to change to the application directory:
cd kenzan/million-song-library/msl-pages
-
Enter the following command to start the application front end:
npm run full-dev
Wait for the front end to start up — this will take a few minutes.
-
Open a new Git Bash window and make sure you are in your home directory.
-
Enter the following command to change to the million-song-library repository:
cd million-song-library
-
Enter the following command to connect to the virtual machine (we don’t need such a long command this time as we already forwarded the necessary ports):
vagrant ssh ubuntu
-
After you are connected to the virtual machine, enter the following command to change to the application directory:
cd kenzan/million-song-library/msl-pages
-
Enter the following command to start the server instances:
sudo npm run serve-all
Wait for the server instances to start up — again, this will take several minutes.
-
Open a Web browser and point it to: msl.kenzanlabs.com:3000
The Million Song Library home page displays. (If you don’t see data right away, wait a couple of minutes and then refresh the page.)
Million Song Library Home Page
Now that the Million Song Library demo is working, here are some fun things to try:
-
Click Register to register for an account.
-
Click the labels to switch the view between Songs, Albums, and Artists.
-
Click a genre or star rating on the left to filter songs. (Click the x to clear a filter.)
-
Mouse over a song and click Add to library to add it to your music library (you must be logged in).
-
Click the stars below a song to rate it (you must be logged in).
8.6. Stop the MSL Demo
Done having fun for now? Here’s how to terminate the virtual machine:
-
Open a new PowerShell window and make sure you are in your home directory.
-
Enter the following command to change to the million-song-library repository:
cd million-song-library
-
Enter the following command to destroy the virtual machine:
vagrant destroy ubuntu -f
The virtual machine is deleted. If you want to run the demo again, just follow this procedure to create a new Million Song Library virtual machine any time.
9. Manual Setup
Are you the type who likes to see all the magic that happens behind the scenes? Or maybe you prefer that things are installed on your system just so. In either case, you can can manually install the Million Song Library on a Mac or Linux system without using the automated setup script. Just follow the steps in this section.
| If you want to get up and running as quickly as possible, using the automated setup script is the easiest way to install the Million Song Library demo on a Mac or Linux system. Or use Vagrant to automate setup on a Windows system. |
9.1. Get Your System Ready
Before setting up the demo, make sure all of the following tools are installed on your system and are at the required version or higher.
This procedure assumes that you’re running the Million Song Library demo on Mac OS X 10.11 (El Capitan) or Ubuntu 14.04 (Trusty Tahr). Things might work a little differently on other operating systems.
Entering Commands
We tested these steps using the default Terminal applications on Mac OS X and Ubuntu.
Where is My Home Directory?
For this process, it’s important to know where your home directory is. Your home directory is the directory that has the same name as your user name.
For example, if the user name you use to log on to your system is jdoe, your home directory is named jdoe. In Terminal, your home directory is represented by the tilde (~) sign. For example, the path ~/stuff points to a directory named stuff located inside your home directory.
- Java 1.8 JDK
-
To check the installed version, use the
java -version
command. If necessary, install the Java 1.8 JDK. - Git
-
To check the installed version, use the
git --version
command. If necessary, install Git. - Maven 3.3.9 or later
-
To check the installed version, use the
mvn -version
command. If necessary, set up Maven. - Node 0.12.x or later
-
To check the installed version, use the
node --version
command. If necessary, install Node. - npm 2.7.x or later
-
To check the installed version, use the
npm -version
command. If necessary, install Node, which includes npm. - Bower
-
To check the installed version, use the
bower --version
command. If necessary, install Bower.
If you get a permissions error, use sudo bower --version instead.
|
- RubyGem
-
To check the installed version, use the
gem --version
command. If necessary, install RubyGem for Mac or Linux. - AsciiDoctor
-
To check the installed version, use the
asciidoctor --version
command. If necessary, install AsciiDoctor. - Cassandra
-
-
Download Cassandra 2.1.11 (Community version) from the Datastax download site.
Make sure to download the correct file: dsc-cassandra-2.1.11-bin.tar.gz -
Create a new directory named cassandra inside your home directory.
-
Move the dsc-cassandra-2.1.11-bin.tar.gz file into the new cassandra directory, and then extract the contents into the cassandra directory.
-
- Apple Xcode (Mac OS X Only)
-
On Mac OS X, some necessary command line tools aren’t installed by default, so to get them you’ll need to install the latest version of Xcode from the Mac App Store.
After installation, make sure to open Xcode, accept the license agreement, and (if prompted) install any additional tools. - Enable Root User (Mac OS X Only)
-
The setup script requires the use of
sudo
to run certain commands as root. However, on Mac OS X, the root user is not enabled by default. You’ll need to enable the root user so that the setup script will run properly.
To enable the root user, just follow these instructions.
9.2. Clone the MSL Repository
-
Make sure you are in your home directory.
-
Clone the million-song-library repository:
git clone https://github.com/kenzanmedia/million-song-library
-
Change directories to the cloned repository:
cd million-song-library
-
Initialize the repository and submodules:
git submodule init && git submodule update
9.3. Set up the Client
-
Change to the msl-pages directory inside the million-song-library directory
cd msl-pages
-
Download and install all dependencies:
npm install && bower install
-
Install Protractor and Selenium WebDriver globally:
npm install -g -y protractor && npm install -g -y selenium-webdriver
9.4. Set up the Server
-
Change to the server directory inside the million-song-library directory:
cd ../server
-
Run the Maven file to set up the server:
mvn clean compile
9.5. Set up Cassandra
-
Open a new Terminal window.
-
Start Cassandra:
sh ~/cassandra/dsc-cassandra-2.1.11/bin/cassandra
-
After Cassandra finishes starting, press <Enter> to display the command prompt again.
-
Enter the Cassandra console:
sh ~/cassandra/dsc-cassandra-2.1.11/bin/cqlsh
-
Import data by entering the following commands at the cqlsh> prompt, pressing <Enter> after each command:
SOURCE 'msl_ddl_latest.cql'; SOURCE 'msl_dat_latest.cql';
Wait for the database to import the data — this will take several minutes.
-
Exit the console:
exit
| If you get an error when trying to start Cassandra, it probably means that Cassandra is already running (for example, from running the Million Song Library demo previously). To correct this issue, reboot your system, or stop Cassandra (see Stop the MSL Demo). Then try starting Cassandra again. You should always make sure to stop Cassandra when you’re done working with the Million Song Library demo. |
9.6. Start the MSL Demo
-
Open a new Terminal window.
-
Change to the application directory:
cd ~/million-song-library/msl-pages
-
Start the application front end:
npm run full-dev
Wait for the front end to start up — this will take just a couple of minutes.
-
Open a new Terminal window.
Make sure to open a new Terminal tab or window before starting the servers, to avoid stopping the front end. -
Change to the application directory:
cd ~/million-song-library/msl-pages
-
Start the server instances:
Linux:
sudo npm run serve-all
Mac:
npm run serve-all
Wait for the server instances to start up — again, this will take just a couple of minutes.
-
Open a Web browser and point it to: msl.kenzanlabs.com:3000
The Million Song Library home page displays.
If you don’t see data right away, wait a few minutes and then refresh the page. If data still doesn’t load, try clearing your web browser cache.
9.7. Stop the MSL Demo
-
Switch to the Terminal window where you started the server instances and press Control+C to stop the server.
-
Switch to the Terminal window where you started the application and press Control+C to stop the application.
-
Switch to the Terminal window where you started Cassandra and enter the following command:
ps auwx | grep cassandra
Look at the output from the command and note the first 3–5 digit number that appears in the output. This is the process ID for Cassandra.
-
Enter the following command where pid is the process ID you found (you’ll be prompted for your administrator password):
sudo kill pid
| You don’t need to run the setup script if you want to start the Million Song Library demo again. Just start Cassandra (see Set up Cassandra) and then start the application and server instances (see Start the MSL Demo). |
10. Deploying to AWS
Deploying the Million Song Library demo locally is a great way to see it in action, but a microservices-based application like MSL is really designed for the cloud. Follow the steps in this section to deploy the Million Song Library demo to an AWS EC2 instance.
10.1.
- AWS Account
-
You will need an AWS account to complete this procedure. If you don’t already have one, you can set up an AWS account for free.
| You may incur costs on your AWS account based on your EC2 instance usage while running the Million Song Library demo. Be sure to stop your MSL instance when you’re not using it. |
- Git
-
You will need Git to make a local copy of the million-song-library repository. If you don’t already have it, install Git.
| (Windows only) During Git installation, make sure to select the Use Git from the Windows Command Prompt option. Otherwise you won’t be able use Git commands in PowerShell. |
- Vagrant
-
The easiest way to set up the Million Song Library in an EC2 instance is with Vagrant, a tool for automating the creation of development environments. If you don’t already have it, install Vagrant.
Entering Commands
We tested these steps using the default Terminal applications on Mac OS X and Ubuntu. To enter commands on Windows 10, we recommend running Windows PowerShell as an Administrator.
Where is My Home Directory?
For this process, it’s important to know where your home directory is. Your home directory is the directory that has the same name as your user name.
-
On Mac and Linux, your home directory is represented by the tilde (~) sign in Terminal commands.
-
On Windows, your home directory is: C:\Users\<your_username>\
10.2. Clone the MSL Repository
To edit the Vagrant file, you first need to clone the million-song-library repository to a local directory.
-
Open a new Terminal or PowerShell window and make sure you are in your home directory.
-
Enter the following command to clone the million-song-library repository:
git clone https://github.com/kenzanmedia/million-song-library
-
Enter the following command to change directories to the cloned repository:
cd million-song-library
-
Enter the following command to initialize the repository and submodules:
git submodule init && git submodule update
For Windows, you will need to run the commands separately: first git submodule init and then git submodule update .
|
10.3. Set up Vagrant for AWS
The AWS plug-in enables Vagrant to automtically create EC2 instances using a dummy virtual machine.
-
Enter the following command to install the AWS plug-in for Vagrant:
vagrant plugin install vagrant-aws
-
Enter the following command to download the dummy virtual machine:
vagrant box add dummy https://github.com/mitchellh/vagrant-aws/raw/master/dummy.box
10.4. Generate an SSH Key Pair
Generate a new SSH key pair for use on the EC2 instance.
| On Windows, use the Git Bash terminal app for this step. You can find it uder Start→All apps→Git→Git Bash. |
-
Generate a new key pair:
-
Enter the following command where<EMAIL_ADDRESS>is your email address:
ssh-keygen -t rsa -C<EMAIL_ADDRESS>-
Enter the file path and name where you want to save the key pair. For example:
/Users/jdoe/.ssh/aws_msl
-
Press <Enter> twice to skip entering a passphrase.
-
-
Import the new SSH key pair into your AWS account:
-
Sign in to your AWS EC2 Management Console and switch to your preferred region.
-
Click Key Pairs (under Network & Security).
-
Click Import Key Pair.
-
Click Choose File and select the new public key file (for example, aws_msl.pub).
Alternately, you can open the public key in a text editor, copy the contents, and paste the contents in the Public key contents box.
-
In the Key pair name box, type the name of the key pair (for example, aws_msl).
-
Click Import.
-
10.5. Generate an AWS Key Pair
Generate a new AWS key pair to allow Vagrant to log on to the EC2 instance.
-
In your EC2 Management Console, click Key Pairs (under Network & Security).
-
Click Create Key Pair.
-
In the Key pair name box, type the name of the key pair (for example, kenzan_msl).
-
Click Create to create and download the private key file.
-
Place the private key file in the same location as the SSH key you created. For example: ~/.ssh/kenzan_msl.pem
10.6. Generate an AWS Access Key
Generate a new AWS access key to allow Vagrant to authenticate with AWS.
-
In your EC2 Management Console, click your user name, and then click Security Credentials.
-
Under Access Keys, click Create New Access Key.
-
Click Download Key File to download the access key file.
-
Place the access key file in the same location as the other keys you created. For example: ~/.ssh/rootkey.csv
| The access key file is a CSV file that contains values for two keys: AWSAccessKeyId and AWSSecretKey. You will need these two key values in a later step. |
10.7. Create a Security Group
Create a new AWS Security Group to allow inbound access to the EC2 instance.
-
In your EC2 Management Console, click Security Groups (under Network & Security).
-
Click Create Security Group.
-
In the Security group name box, type the name of the security group (for example, kenzanmsl).
-
In the Description box, type a description for the security group (for example, Million Song Library Demo).
-
On the Inbound tab, click Add Rule.
-
For Type, select All traffic.
-
For Source, select Anywhere.
-
Click Create.
10.8. Edit the Vagrant File
The Vagrant file contains all of the information needed to set up and configure the EC2 instance. You need to edit the Vagrant file to add information about your AWS account — this lets Vagrant know where to set up the instance.
-
In a text editor, open the file named Vagrantfile located in the million-song-library directory (in your home directory).
-
Replace the following variables with the correct values:
Variable Value <<PATH_TO_SSH_PRIVATE_KEY>>
File path and name for your private SSH key. For example: ~/.ssh/aws_msl
<<PATH_TO_SSH_PUBLIC_KEY>>
File path and name for your public SSH key. For example: ~/.ssh/aws_msl.pub
<<KEY_PAIR_NAME>>
Name of your AWS key pair. For example: kenzan_msl
<<PATH_TO_YOUR_AWS_KEY.pem>>
File path and name for your private AWS key. For example: ~/.ssh/kenzan_msl.pem
<<AWS_ACCESS_KEY_ID>>
AWSAccessKeyId value from the rootkey.csv access key file you downloaded.
<<AWS_SECRET_ACCESS_KEY>>
AWSSecretKey value from the rootkey.csv access key file you downloaded.
<<SECURITY_GROUP_WITH_SSH_AND_HTTP_PORTS_ENABLED>>
Name of the AWS security group you created for use with the MSL demo. For example: kenzanmsl
aws.region = "us-west-2"
If necessary, replace the region code with your preferred AWS region.
aws.instance_type="t2.small"
Replace the instance type with the desired type. We recommend t2.large or better to avoid running out of resources.
-
Save and close the Vagrant file.
10.9. Provision an EC2 Instance
After you finish editing the Vagrant file, you’re ready to use Vagrant to create and provision the EC2 instance with the Million Song Library Demo.
-
Open a new Terminal or PowerShell window and make sure you are in your home directory.
-
Enter the following command to change to the million-song-library repository:
cd million-song-library
-
Enter the following command to start the provisioning process:
vagrant up prod
-
Wait for the provisioning process to complete (about 30 minutes).
If you get an error stating that the .gitconfig file does not exist, enter the command touch ~/.gitconfig to create one, and then run vagrant up prod again.
|
10.10. Edit the Hosts File
While you’re waiting for the provisioning process to complete, take a few minutes to update the hosts file on your local system. This will allow you to access the Million Song Library application using a Web browser.
-
In your EC2 Management Console, click Instances to view all running instances. You should see the Production-aws instance created by Vagrant.
-
Copy the Public IP of the instance.
-
In a text editor, open the hosts file on your computer. The location of the hosts file depends on your operating system:
Operating Syystem Location of hosts file Mac
/private/etc/hosts
Linux
/etc/hosts
Windows
C:\Windows\System32\Drivers\etc
-
Add the following line to the hosts file where xxx.xxx.xxx.xxx is the public IP address of the EC2 instance.
xxx.xxx.xxx.xxx msl.kenzanlabs.com
-
Save and close the hosts file.
10.11. Start the MSL Demo
Provisioning is complete — now it’s time to start the Million Song Library demo.
-
Open a new Terminal or PowerShell window and make sure you are in your home directory.
-
Enter the following command to connect to the EC2 instance:
vagrant ssh prod
-
After you are connected to the instance, enter the following command to change to the application directory:
cd million-song-library/msl-pages
-
Enter the following command to start the application front end:
npm run deploy
Wait for the front end to start up — this will take just a couple of minutes.
-
Open a new Terminal or PowerShell window and make sure you are in your home directory.
Make sure to open a new Terminal tab or window before starting the servers, to avoid stopping the front end. -
Enter the following command to change to the application directory:
cd million-song-library/msl-pages
-
Enter the following command to start the server instances:
npm run serve-all
If your EC2 instance is low on free memory, you can start up just the core server instances by running the npm run catalog-edge-server
command instead.Wait for the server instances to start up — again, this will take just a couple of minutes.
-
Open a Web browser and point it to: msl.kenzanlabs.com:3000
The Million Song Library home page displays.
If the Million Song Library home page does not display, make sure that you allowed all inbound traffic in your AWS security group and that you correctly edited the hosts file. If you don’t see data right away, wait a few minutes and then refresh the page. If data still doesn’t load, try clearing your web browser cache.
Now that the Million Song Library demo is working, here are some fun things to try:
-
Click Register to register for an account.
-
Click the labels to switch the view between Songs, Albums, and Artists.
-
Click a genre or star rating on the left to filter songs. (Click the x to clear a filter.)
-
Mouse over a song and click Add to library to add it to your music library (you must be logged in).
-
Click the stars below a song to rate it (you must be logged in).
10.12. Stop the MSL Demo
Done having fun for now? Here’s how to terminate the EC2 instance:
-
Open a new Terminal or PowerShell window and make sure you are in your home directory.
-
Enter the following command to change to the million-song-library repository:
cd million-song-library
-
Enter the following command to destroy the EC2 instance:
vagrant destroy prod -f
The EC2 instance is terminated. If you want to run the demo again, just follow this procedure to create a new Million Song Library instance any time.


Welcome, @Model.Username!
Your channels
@Model.Collection.FollowedChannels(@Item)
Suggested
@Model.Collection.SuggestedChannels(@Item)
See Other
@Model.Collection.OtherChannels(@Item)
Welcome, @Model.Username!
Your channels
@Model.Collection.FollowedChannels(@Item)
Suggested
@Model.Collection.SuggestedChannels(@Item)
See Other
@Model.Collection.OtherChannels(@Item)


#include <stddlg.h>
Inheritance diagram for TDirListBox::
Public Methods | |
| TDirListBox (const TRect &bounds, TScrollBar *aScrollBar) | |
| ~TDirListBox () | |
| virtual void | getText (char *dest, short item, short maxLen) |
| virtual Boolean | isSelected (short item) |
| virtual void | selectItem (short item) |
| void | newDirectory (const char *aList) |
| virtual void | setState (ushort aState, Boolean enable) |
| TDirCollection* | list () |
Static Public Methods | |
| TStreamable* | build () |
Static Public Attributes | |
| const char* | pathDir |
| const char* | firstDir |
| const char* | middleDir |
| const char* | lastDir |
| const char* | graphics |
| const char* const | name |
Protected Methods | |
| TDirListBox (StreamableInit) |
By default, these are displayed in a single column with an optional vertical scroll bar.
Definition at line 1159 of file stddlg.h.
|
|
Calls TListBox::TListBox(bounds, 1, aScrollBar) to create a single-column list box with the given bounds and vertical scroll bar.
|
|
|
Calls its base destructor to dispose of the list box.
|
|
|
Undocumented. |
|
|
Called to create an object in certain stream-reading situations. Reimplemented from TListBox. |
|
|
Grabs the text string at index `item' and copies it to `dest'. Reimplemented from TListBox. |
|
|
Returns True if `item' is selected, otherwise returns False. Reimplemented from TListViewer. |
|
|
Returns a pointer to the TDirCollection object currently associated with this directory list box. Reimplemented from TListBox. |
|
|
Deletes the existing TDirEntry object associated with this list box and replaces it with the file collection given by `aList'. The first item of the new collection will receive the focus. |
|
|
Undocumented. Reimplemented from TListViewer. |
|
|
By default, calls the ancestral TListBox::setState(). Reimplemented from TListViewer. |
|
|
Undocumented. |
|
|
Undocumented. |
|
|
Undocumented. |
|
|
Undocumented. |
|
|
Undocumented. Reimplemented from TListBox. |
|
|
Undocumented. |


Address: 127.127.35.u
Reference ID: PCF
Driver ID: PCF
Parallel Port: /dev/pcfclocks/u or /dev/pcfclocku
This driver supports the parallel port radio clock sold by Conrad Electronic under order numbers 967602 and 642002. This clock is put between a parallel port and your printer. It receives the legal German time, which is either CET or CEST, from the DCF77 transmitter and uses it to set its internal quartz clock. The DCF77 transmitter is located near to Frankfurt/Main and covers a radius of more than 1500 kilometers.
The pcfclock device driver is required in order to use this reference clock driver. Currently device drivers for Linux and FreeBSD are available.
This driver uses C library functions to convert the received timecode to UTC and thus requires that the local timezone be CET or CEST. If your server is not located in Central Europe you have to set the environment variable TZ to CET before starting ntpd.
Each timecode is written to the clockstats file in the format YYYY MM DD HH MI SS.


USD/INR: Risk-off may weigh over Indian Rupee
by Omkar Godbole- Rupee may be offered due to a risk-off tone in the global markets.
- The Indian government's focus on fiscal prudence may restrict losses in INR.
- USD/INR has established 71.44 as key support in the last three days.
The Indian Rupee (INR) may come under pressure on Wednesday, courtesy of the renewed US-China political tensions and the resulting risk-off tone in the equity markets. The downside, however, may be capped by the Indian government's focus on fiscal prudence.
The futures on the S&P 500 fell 0.25% and the AUD/JPY pair, a risk barometer, shed 30 pips in Asia after China's foreign ministry threatened retaliation over the decision by the US Senate to pass legislation aimed at protecting the human rights in Hong Kong.
Further, President Trump on Tuesday said that the US will raise tariffs on China if the world's second-largest economy does not agree to a trade deal that he wants.
The downside in the INR, however, could be restricted by the Indian government's decision to keep intact the fiscal deficit target of 3.3% of gross domestic product (GDP) for the current financial year despite the slowdown in the economy.
Also, China's rate cut may restrict losses in the equities, helping the INR avoid big losses. China reduced the one-year loan prime rate to 4.15% from 4.20% and the five-year rate to 4.80% from 4.85%.
USD/INR closed at 71.72 on Tuesday, representing a 0.17% drop on the day. Sellers have failed to penetrate the area around 71.44 over the last three days.
Technical levels
USD/INR
| Overview | |
|---|---|
| Today last price | 71.825 |
| Today Daily Change | -0.0055 |
| Today Daily Change % | -0.01 |
| Today daily open | 71.8305 |
| Trends | |
|---|---|
| Daily SMA20 | 71.2372 |
| Daily SMA50 | 71.1575 |
| Daily SMA100 | 70.7191 |
| Daily SMA200 | 70.2134 |
| Levels | |
|---|---|
| Previous Daily High | 72.18 |
| Previous Daily Low | 71.611 |
| Previous Weekly High | 72.37 |
| Previous Weekly Low | 70.97 |
| Previous Monthly High | 71.79 |
| Previous Monthly Low | 70.6425 |
| Daily Fibonacci 38.2% | 71.8284 |
| Daily Fibonacci 61.8% | 71.9626 |
| Daily Pivot Point S1 | 71.5677 |
| Daily Pivot Point S2 | 71.3048 |
| Daily Pivot Point S3 | 70.9987 |
| Daily Pivot Point R1 | 72.1367 |
| Daily Pivot Point R2 | 72.4428 |
| Daily Pivot Point R3 | 72.7057 |


Speech
The Education Secretary Nicky Morgan addresses the ASCL (Association of School and College Leaders) conference.
Thank you, Brian [Lightman, ASCL General Secretary], it’s a pleasure to be here.
It’s now been almost 9 months since I took on the job of Education Secretary and if I’ve taken one thing away from my time so far, and I very much hope this is just the beginning of my time as Education Secretary, it’s the fact that everyone has an opinion on education.
Parents, grandparents, teachers, heads, pupils, governors, families - seemingly the entirety of Twitter - and one thing is also clear, there’s often very little agreement between them!
I have to say, I admire the lengths that my Argentinian counterpart has gone too this week to call off teacher strikes, by giving every teacher his mobile phone number and inviting them to call him directly with questions, comments or complaints. I know that Brian welcomed his move, but for those of you who may be getting ideas, let me tell you now, I’m all for learning from the best - but this may be the one international initiative I may not be adopting that quickly.
But in all seriousness, I think you’d be hard pressed to find another policy area where so many different people and groups have an opinion about what should be done and how it should be achieved.
Partly this is because, almost uniquely among the public services, all of us have experienced one form of education or another. But I think it’s also about something deeper - our nation’s recognition of just how important a good education is, and the truly life-transforming potential it holds.
No one recognises that more than you. Because the one thing that has shone out to me, from the 700 teachers I’ve spoken to on school visits across the country since taking this role, is not only the dedication of our heads and teachers, but how much they cherish the work they do to open young minds and change lives.
Let’s be honest, you haven’t had it easy, you never have, and this government has expected more from school leaders than almost any other - you have been at the forefront of our plan for education, delivering significant changes to how and where our pupils are taught.
But away from the rhetoric, away from the personality-obsessed politics of the Westminster bubble, school leaders like you have been the ones translating our policies into concrete action on the ground.
You haven’t agreed with everything we’ve done, and we’d have been surprised if you had.
From my Teacher Direct sessions, which for those of you who don’t know are question-and-answer events I’ve instituted with teachers and heads around the country, I know that there have been concerns. But what I hope, is that in those areas where you’ve told us we haven’t got it quite right, we’ve been able to work with you to get it better.
Take for instance the issue of assessment levels.
I remain convinced that school leaders are best placed to come up with assessment systems which work for them, and their students - and crucially are understandable for parents.
We know that assessment levels often weren’t trusted by secondary schools because they didn’t really reveal what children knew and forced them to move on to new material before they were ready, resulting in serious gaps in their knowledge.
So I’m pleased that we’re moving to a system that schools are free to develop their own systems of assessment of ‘real’ things - like how well a child can read and the specifics of what they know and can do in maths. Things that can inform better teaching and provide clearer, more incisive evidence of attainment and progression.
But I’m also aware that the transition hasn’t been easy. In fact, levels come up in almost every one of my question and answer sessions.
So to address those concerns we recently announced the launch of a commission, led by teachers, on assessment without levels to help schools learn from the best and provide that extra support. Because, and let me be crystal clear on this point, I want the changes we’ve made to be changes that allow teachers to do what they do best, ie teach.
The good news is that these tricky issues seem to be the exception, rather than the rule. And the general picture I find on school visits, is that school leaders are relishing the new freedoms we’ve offered to them, relishing the fact that they are being treated as professionals, and relishing the fact that this government recognises they know best what’s right for their pupils.
I’ll go on to talk about my reaction to your blueprint shortly, but the message that stands out from that document so clearly, is that the profession wants to see those freedoms go further, faster and deeper.
That is a direction of travel to which I’m wholly prepared to commit.
Because the results of trusting professionals and giving schools greater freedom speak for themselves:
This is what you have achieved.
As a result, every day we are moving closer towards tackling that ‘soft bigotry of low expectations’ towards a society where no child is held back by the circumstances of their birth.
And towards a country where every young person leaves school equipped with the knowledge, skills and values they need to truly succeed in modern Britain.
This is just the beginning and you know as well as I do, how much there is still left to do:
I want to reassure you about what that means in practice, it doesn’t mean 5 years of constant upheaval or constant change.
What it does mean is ensuring that the impact of those changes reaches every part of the country, every child, every family and every community.
Both the Prime Minister and I are very clear that the changes we’ve made need time to bed in and take root. ASCL ’s ‘ Blueprint for self-improving system ’ calls for an accountability framework that has been in place for the term of the government, a vision that I share.
From the introduction of the National Teacher Service, to requiring every young person to master the basics in primary school and boosting the number of young people taking the EBacc, there is a common thread: that a combination of high expectations for young people and unleashing teachers’ freedom can produce fantastic results.
What we haven’t achieved is excellence everywhere. In particular in some of our coastal areas and small towns educational performance is a scandal.
Take just one example:
So the priority will be to work with you to spread excellence across the system, to make sure we get our best teachers everywhere, and that we give every young person the skills they need to compete not just with their peers in Britain, but across the globe.
I know it’s a vision that ASCL shares. Government standing back and trusting the profession, and instead focusing our attention on addressing those strategic issues that threaten future success.
It’s now a hackneyed phrase - but is undeniably true that no system can be better than the teachers that teach in it and the leaders who lead it.
It’s why we’ve given you more input in the recruitment and training of teachers through our School Direct programme - and looked to attract top talent through initiatives like Teach First and the £67 million push to boost maths and science teaching that the Prime Minister has just announced.
It’s the reason we’re supporting high-quality professional development through projects like our exchange with Shanghai, which has seen teachers from this education powerhouse teaching maths in primary schools across England.
And why we’re encouraging school leaders to work together more closely to share expertise and help others improve through, for example, our growing network of over 650 teaching schools and system leaders such as national leaders of education, specialist leaders of education, and national leaders of governance.
And we’ve worked with the professionals to develop new headteacher standards that aren’t simply a baseline, a hurdle for school leaders to clear. They’re aspirational and ambitious, they don’t specify the acceptable, but instead describe the exceptional.
One of my priorities has been to make sure that teachers and school leaders can spend more of their time doing what they do best- inspiring young minds. Ensuring that teachers can spend more time in the classroom and that you as school leaders don’t face losing your best and brightest staff.
That’s what underpinned our workload challenge, which generated one of the largest number of responses to a consultation my department has seen. It created quite a workload challenge for us too.
As you know, we’ve announced some important actions in response to this - such as a commitment to give schools more notice of significant changes to the curriculum, qualifications and accountability.
I hope these will go some way to eliminating the bureaucracy and duplication of work that’s taking teachers away from teaching.
I know the results haven’t pleased everyone, some were clearly expecting a silver bullet that simply wasn’t there.
But what I am clear about is that this is very much the start of the journey - not the end - and we will continue to listen and work with you to tackle these issues.
Accountability is clearly one of those areas where we have more work to do to ensure the way we inspect our schools is fair and proportionate.
I’m pleased that Ofsted is continuing to look at ways to improve the quality of inspections, ensuring that inspections are focused on providing information for parents and families, rather than driving the behaviour of schools and teachers.
Like you, I’m excited about the potential that open data has too, as the blueprint puts, it ‘further empowers parents and the community’ and I’m convinced the more we do to unlock that data and make it accessible to families, the less we’ll have to rely so heavily on external inspection.
The theme of this conference is trust, and over the course of the conference Peter [Kent, ASCL President], Brian and many of the delegates have made the powerful case for trusting school leaders and trusting the profession. We accept that case wholeheartedly.
It’s trust that inspired the Prime Minister and I to announce yesterday that we’d do our bit to facilitate the creation of a college of teaching: fully independent of government and a champion not just for the professional development of teachers but also the development of their practice on the ground - a key part of the first element of your blueprint, ‘refining, challenging and developing the teaching profession’.
Alongside this we’ve committed up to £5 million to establish the first phase of a new professional development fund, to support more high-quality CPD programmes, delivered by our growing network of teaching schools. Another marker of the profession taking its destiny into its own hands.
Like you, I also share a vision for even greater collaboration between schools, and I hope that over the course of the next Parliament we see even greater school collaboration through academy chains and trusts. Because whether it’s Ark in Portsmouth, Perry Beeches in Birmingham or Harris in London, we know that working together through multi-academy trusts, schools can achieve truly extraordinary outcomes for young people.
Because where I think the blueprint has hit the nail on the head is in its insistence that ‘self-improvement’ means something more than simply ‘school-led’. ‘Self-improvement’ demands a mature and measured approach to challenging practice, to driving continuous improvement and to working together to push boundaries.
Of course there will still be areas where we will still disagree. It would be easy for me 6 weeks before an election to simply say I agreed with everything that the blueprint lays out. Much of which I do, but that wouldn’t be a very good basis for the kind of dialogue that your blueprint demands.
And there is one area where I depart from the blueprint - in its suggestion of taking control of the curriculum away from ministers. Because it’s my belief that what our children learn in schools must be something that is decided by democratically elected representatives.
That isn’t because I think I understand algebra any better than you do, or that Nick Gibb understands phonics any better than the teachers that teach it, although don’t tell him that. But because I think that parents should be able to hold us to account for the decisions we make about what their children are learning and what they’re not; and the surest way to make sure they can do that is at the ballot box.
That doesn’t mean that we fly blind. As I hope you’ve seen throughout the process of reviewing curriculum and qualifications we’ve involved experts at every step of the way - but those taking the final decision must remain accountable to the public at large.
But as a whole, your blueprint for the future of our school system outlines a vision that I share. Of a confident teaching profession, of practice driven on the ground, of government as the supporter and facilitator of progress - but not the director.
 I very much hope that the electorate will give me the chance to carry on.
But regardless of the result in 46 days’ time, I want to leave you with this message.
On every school visit, in every corner of the country, I am truly inspired by the work you do to change young people’s destinies, opening their eyes to the future, their minds to the possibilities.
I will always be in awe of your skill and dedication, of your determination to secure the best for your pupils.
And I, and the Prime Minister, and for that matter thousands of parents and families across the country will always be grateful, not just for what you do day to day, but for the leading role you have played in delivering our plan for education.
Those million extra pupils in ‘good’ or ‘outstanding’ schools are your pupils, the record levels of students going on to university or in to an apprenticeship are your students. And those children who are leaving school today prepared and ready to succeed in modern Britain are your children. You should be incredibly proud.
Thank you for everything you do.
Published: 21 March 2015


Next: Sharpsign B, Previous: Sharpsign Colon, Up: Sharpsign
#.foo is read as the object resulting from the evaluation of the object represented by foo. The evaluation is done during the read process, when the #. notation is encountered. The #. syntax therefore performs a read-time evaluation of foo.
The normal effect of #. is inhibited when the value of *read-eval* is false.
In that situation, an error of type reader-error is signaled.
For an object that does not have a convenient printed representation, a form that computes the object can be given using the #. notation.


JavaScript is disabled on your browser.
Skip navigation links
Package
Class
Tree
Index
Help
Prev Letter
Next Letter
Frames
No Frames
All Classes
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
Q
R
S
T
U
V
W
X
Y
Z
B
background(Color)
- Method in class JProcessing.
Processing
background(Color, value)
- Method in class JProcessing.
Processing
background(value)
- Method in class JProcessing.
Processing
background(value, value)
- Method in class JProcessing.
Processing
background(value, value, value)
- Method in class JProcessing.
Processing
background(value, value, value, value)
- Method in class JProcessing.
Processing
background(BufferedImage)
- Method in class JProcessing.
Processing
BACKSPACE
- Static variable in class JProcessing.
Processing
The backspace character
BASELINE
- Static variable in class JProcessing.
Processing
Text alignment -- default text alignment setting
beginDraw()
- Method in class JProcessing.
Processing.PGraphics
beginShape()
- Method in class JProcessing.
Processing
beginShape(byte)
- Method in class JProcessing.
Processing
BEVEL
- Static variable in class JProcessing.
Processing
Stroke join mode -- joins path segments by connecting the outer corners of their wide outlines with a straight segment.
bezier(double, double, double, double, double, double)
- Method in class JProcessing.
Processing
bezier(float, float, float, float, float, float)
- Method in class JProcessing.
Processing
bezier(double, double, double, double, double, double, double, double)
- Method in class JProcessing.
Processing
bezier(float, float, float, float, float, float, float, float)
- Method in class JProcessing.
Processing
bezierVertex(value, value, value, value, value, value)
- Method in class JProcessing.
Processing
binary(int)
- Static method in class JProcessing.
Processing
binary(Color)
- Static method in class JProcessing.
Processing
binary(Color, int)
- Static method in class JProcessing.
Processing
binary(int, int)
- Static method in class JProcessing.
Processing
blue(int)
- Method in class JProcessing.
Processing
blue(Color)
- Method in class JProcessing.
Processing
borderHeight
- Variable in class JProcessing.
Processing
Height of GUI window's borders in total (Read-Only)
borderWidth
- Variable in class JProcessing.
Processing
Width of GUI window's borders in total (Read-Only)
BOTTOM
- Static variable in class JProcessing.
Processing
Text alignment -- align text to the bottom
brightness(int)
- Method in class JProcessing.
Processing
brightness(Color)
- Method in class JProcessing.
Processing
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
Q
R
S
T
U
V
W
X
Y
Z
Skip navigation links
Package
Class
Tree
Index
Help
Prev Letter
Next Letter
Frames
No Frames
All Classes


及时清理不再使用的代码段或配置信息。
说明：对于垃圾代码或过时配置，坚决清理干净，避免程序过度臃肿，代码冗余。
Positive example: For codes which are temporarily removed and likely to be reused, use /// to add a reasonable note. public static void hello() { /// Business is stopped temporarily by the owner. // Business business = new Business(); // business.active(); System.out.println("it's finished"); }


Clock Speed:
0
/20
Learn
Pause
Sound
Welcome User! This is a Training Phase. Press Start Training to begin.