Artificial Intelligence based chip design
- Artificial Intelligence based chip design: Transformers on Chip (part-1)
- Artificial Intelligence based chip design: Chip Design (part-2)
Introduction
We have taken a traditional (TTS)Text-to-speech and (STT)Speech-to-text system and spun it on an FPGA microchip. In the pipeline of a traditional TTS system, there is a Seq-to-Seq LSTM layer rearing its “ugly latency head”. It takes approximately tens of thousands of iterations of this network to generate a second of audio.Sometimes this latency is not acceptable. Sometimes, this latency is not acceptable. Latency of these systems can be majorly attributed to the LSTMs, the technology of the day. LSTMs still have their uses, but for speech-based latency sensitive systems, they have limitations. LSTMs have played their part in the Deep Learning history and it is time for them to pass on the baton to the new King, The Transformers.
STT(Speech-to-Text)
Firstly, we took the STT based on Conformer (a transformer variant) and rewrote it using Trax.
Why Trax? Well, it has an optimizing Just-In-Time compiler for Machine learning. We have extended that for FPGA. But more on that later.
It was a huge improvement on the LSTM variant. While there was a marked improvement, it wasn’t enough. And this was running on a 2020 GPU.
Transformers are essentially ((MHSA)Multi-headed-Self-Attention) and a (FFM)feed forward Module. Both these are matrix multiplications of grand and varying scale. The Conformer has an additional CM(Convolutional Module) which is a matrix filter operation.
TTS(Text-to-Speech)
Secondly, we took the TTS based on the transformer and did exactly the same. Again it was a decent improvement on the LSTM version. But, not enough. Again, this was running on a 2020 GPU.
GPUs are great at handling the “grand” part despite some transformers having billions to even trillions of parameters.
- The “varying” part is proving extremely tricky for GPUs. Both MHSA and FFM have varying parallelism requirements(from one model to another).
- Also, the standard precision leads to lots of wastage energy. GPU’s SIMD/SIMT lock-step style execution model makes matters worse. Lock step style execution is prohibitive in terms of energy wastage.
- Last but not least, the extra control structure on the GPU is not useful(wasteful) for matrix multiplication. GPU is a general-purpose device and for something as specific as tuning a massive matrix multiplication it is found wanting. It is not a natural fit would be the right phrase.
- All of the above contribute to tons of electricity, produce a lot of heat, and use fans for cooling.
- And a large number of environments where we apply deep learning like speech devices, self-driving cars, production lines, etc, are not agreeable to it.
- This also makes the maintenance and life expectancy of a GPU an issue.
FPGAs as modern digital ShapeShifters
The CPUs and especially GPUs are nearing their transistor limit. The world is moving toward specialized hardware to meet the exponentially growing demand for computers:
- Field-programmable gate arrays (FPGAs) are reconfigurable computer chips that can be programmed to implement any digital hardware circuit.
- FPGAs consist of an array of different types of programmable blocks (logic, IO, and others) that can be flexibly interconnected using prefabricated routing tracks with programmable switches between them.
- The bit-level reconfigurability of FPGAs enables implementation of the exact hardware needed for each application (e.g. datapath bit-width, pipeline stages, number of parallel compute units, memory subsystem, etc.) instead of the fixed one-size-fits-all architecture of CPUs or GPUs.
- Consequently, they can achieve higher efficiency than CPUs or GPUs by implementing instruction-free streaming hardware
- FPGA-RAM
- An FPGA BRAM(BlockRAM) consists of an SRAM-based memory core, with additional peripheral circuitry to make them more configurable for multiple purposes and to connect them to the programmable routing FPGA vendors can add circuitry that allows designers to repurpose the LUTs(Look Up Tables) that form the logic fabric into additional RAM blocks.
- DSP Blocks
- With the prevalence of multipliers in FPGA designs from key application domains, and their lower area/delay/power efficiency when implemented in soft logic, they quickly became a candidate for hardening as dedicated circuits in FPGA architectures.
Optimizing on Hardware Accelerator
The idea here is to hardware accelerate the huge matrix multiplication which is the core of Transformers. Given the size of the matrices in transformers ( a few billion parameters in 2020), there is practically no chance of them fitting inside a top end FPGA board, not for at least the next decade. So the trick is to chop these huge matrices being multiplied into smaller tiles and reuse these tiles as much as possible before moving on to the next tile. One of the oldest tricks in a new Avatar.
There are 4 key points of FPGA Hardware Optimization
- Play to FPGAs strength, no extra control logic
- Minimize access to OFF-chip RAM by having multi level on chip buffer made of BRAM and LUTRAM
- Maximize the compute parallelism (PE)ProcessingElements
- And the most vital, balance the above 2.
- PEs must be well fed and not starving
- Balance cached data with streaming( weights(Mul1) may be cached more and the input will be streamed more(Mul2))
Optimizing on Hardware Accelerator(MAC Style)
The hardware mapping is the most fabulous part carried out by our mapper. It takes the code below and unrolls the MaxMul loop into the Hardware.
- There are 2 Styles
- MAC Style(Multiply Accumulator)
- For a 4X4 matrix a 2X2 PE array as shown will take 24 cycles give or take.
- For a 4X4 matrix a 4X4 PE array as shown will take 6 cycles give or take.
- SA Style (Systolic Array)
- For a 4X4 matrix a 2X2 PE array as shown will take 24 cycles give or take.
- For a 4X4 matrix a 4X4 PE array as shown will take 6 cycles give or take.
- We retrofit this mapper to trax.
Optimizing on Hardware Accelerator(Systolic Array Style)
Same as above except the microarchitecture uses systolic array design with The PEs being pipelined. Follow the color coding especially the data movement(light and dark blue boxes and arrows) into the PEs(Processing Elements) and data forwarding among PEs. The NOC(Network On Chip) architecture which is not shown, is slightly different for both the MAC Style(Multiply Accumulator) and the SA Style (Systolic Array).
Variables in the design
The SA Style (Systolic Array) works better for bigger tiles when it’s pipelines are fully fed for longer durations. But not too big as the NOC feeding the buffers has its own limitations. The size of the tile and indeed the style of the microarchitecture( MAC or SA) are one of the many design variables that need to be configured for best results. These in turn depend on quite a few more design variables, some of them are due to software constraints (Matrix size etc.) and others are hardware constraints (BRAM etc). Here are a few important design variables.
- The SGB(size of Global Buffer) based on available BRAM and LUTRAM.
- More LUTRAM would mean bigger buffers, better reuse, but fewer PEs and lesser parallelism.
- The LGB(Layout(Locality) of GLobal Buffer)
- The size of the MT (Matrix Tile) and PET(PE Tile)
- The size of RF(RegisterFile)
- As a general rule, bigger/multi-level caching(SGB/NOC/RF are like L3/L2/L1) would better lookup performance but increase data replication and eat into logic space(PEs)
- The layout of NOC(Network on Chip)
- With 100s of MBs to 1000 GBs of matrix sizes to choose from in modern transformers, myriad different Accelerator boards, Shape shifting configuration options on these boards, all the above variables to choose from, the configuration space is a serious problem of plenty.
- A general configuration done by a domain expert(on Transformers and FPGA board) will give us decent improvement over a software only implementation but, nowhere near what the hardware is capable of unless the configurations are done by either Ramanujam or Tesla(Only these 2, others were normal).
Summary
Migrating compute that requires extreme parallelism, or compute that can benefit from custom data path or precision should be moved to FPGA chips. Infact this is FPGA’s backyard. The overall throughput can be a few hundred times better. Latency can be a bit tricky, but still can be a lot better with the flexibility that FPGA chips offer.
References
- What every programmer should know about memory: (This is a definitive 9 part(the links for the rest of the parts are embedded in the first one.) article on how the hardware works and, how software and data structure design can exploit it. It had a huge impact on the way I thought and still do think about design. The article first came out in 2007 but it is still relevant which is proof that basics don’t change very often.)


Demonstration of CodeMirror's Groovy highlighter.
Created by eXo Platform SAS (license).
Note that the files for this parser aren't included in the CodeMirror repository, but have to fetched from svn.exoplatform.org:


This Website aims to help and give tech supports about a certain problem concerning computers or any software. We aim to help and also intertain people who is very well interested about the new developements of computers and softwares.
The estimated number webpages of this Website is about 20 Webpages. This Website may contain more web pages than the estimated number due to the features of this Website.
This Website is for any people who is interested about computers and software who wish to be always updated about this field.
This lists contains some of the problems that maybe encountered during designing Web Pages.


Pat's Salmon Merchandise
| Show off your love for Salmon with our new Salmon T-shirt! Comes in various sizes for men, women, and children. | |
| Impress your dinner guests with you own fish shaped cookies! | |
| We have an assortment of salmon magnets! | |
| Love Chinook?  Frame included. |


Frequently Asked Questions
Q: What are Marker Bots made of?
A: The body of the Standard Marker Bot is made out of wood, with galvanized steel rods. We also have Ceramic and Steel Marker Bots, which are a bit pricier.
Q: Will I be able to get Marker Bot in my country?
A: We'll start off by just shipping to Australia, but after a while we should be able to expand to other countries.
Q: How long does Marker Bot take to write a page?
A: The Marker Bot writes at about 1 letter every five seconds. This means that for a 500 word page, it would take around 40 minutes.
Q: How long does it take the Marker Bot to learn someone’s handwriting?
A: It depends on what quality you want, but it usually takes between 2 and 25 minutes.
Q: When will Marker Bot be available?
A: We have not yet finished designing Marker Bot, so it will be at least a year, and probably more, before we are ready to ship.
Q: What colours can I get my Marker Bot in?
A: When you order a Marker Bot, you can pick a colour. We will try our hardest to find the closest colour we can.
Q: How long will my Marker Bot last?
A: Since it is made of high quality parts, it should last a long time. If anything does break, we will happily send any replacement parts.
Q: Can you use any pencil or pen in the Marker Bot?
A: Marker Bot holds the pen using an elastic strap, so any pen or pencil between 4mm and 2cm in diameter will work just fine. Smooth ballpoint pens, felt tip pens, and mechanical pencils work well, but normal pencils don't as they have to be sharpened.
Q: How much will it all cost?
A: The standard Marker Bot costs around $200, although it is better to check our store page
Q: Do people live in Australia? Why ship there?
A: Yes. We do.
Q: Can Marker Bot write multiple pages?
A: Marker Bot can generate and write multiple pages in one go, but you'll have to switch over the paper manually. Marker Bot will sound an alarm to let you know it's finished the page, and wait for you to press continue.
Q: Can I share my handwriting outside the Marker Bot library?
A: Yes. Just copy the style data from the 'Export Style' box (it should look something like "#quality=X;style=XXXXXXXXXX;letterWidths=X+X,X+X,X+X,;
") and share it. To use the style, paste it at the end of the app's url, or into the 'Import Style' box.
Q: Can I help make Marker Bot a success?
A: Yes! Just go to our Kickstarter page, and pledge however much you like. If you're not willing to spend money, you can help spread the word!


Query Tool
The Query Tool is a powerful, feature-rich environment that allows you to
execute arbitrary SQL commands and review the result set. You can access the
Query Tool via the Query Tool menu option on the Tools menu, or through the
context menu of select nodes of the Browser tree control. The Query Tool
allows you to:
- Issue ad-hoc SQL queries.
- Execute arbitrary SQL commands.
- Edit the result set of a SELECT query if it is
updatable.
- Displays current connection and transaction status as configured by the user.
- Save the data displayed in the output panel to a CSV file.
- Review the execution plan of a SQL statement in either a text, a graphical
format or a table format (similar to https://explain.depesz.com).
- View analytical information about a SQL statement.
You can open multiple copies of the Query tool in individual tabs
simultaneously. To close a copy of the Query tool, click the X in the
upper-right hand corner of the tab bar.
The Query Tool features two panels:
- The upper panel displays the SQL Editor. You can use the panel to enter,
edit, or execute a query. It also shows the History tab which can be used
to view the queries that have been executed in the session, and a Scratch Pad
which can be used to hold text snippets during editing. If the Scratch Pad is
closed, it can be re-opened (or additional ones opened) by right-clicking in
the SQL Editor and other panels and adding a new panel.
- The lower panel displays the Data Output panel. The tabbed panel displays
the result set returned by a query, information about a query’s execution plan,
server messages related to the query’s execution and any asynchronous
notifications received from the server.
The SQL Editor Panel
The SQL editor panel is a workspace where you can manually provide a query,
copy a query from another source, or read a query from a file. The SQL editor
features syntax coloring and autocompletion.
To use autocomplete, begin typing your query; when you would like the Query
editor to suggest object names or commands that might be next in your query,
press the Control+Space key combination. For example, type “*SELECT * FROM* ”
(without quotes, but with a trailing space), and then press the Control+Space
key combination to select from a popup menu of autocomplete options.
After entering a query, select the Execute/Refresh icon from the toolbar. The
complete contents of the SQL editor panel will be sent to the database server
for execution. To execute only a section of the code that is displayed in the
SQL editor, highlight the text that you want the server to execute, and click
the Execute/Refresh icon.
The message returned by the server when a command executes is displayed on the
Messages tab. If the command is successful, the Messages tab displays
execution details.
Options on the Edit menu offer functionality that helps with code formatting
and commenting:
- The auto-indent feature will automatically indent text to the same depth as
the previous line when you press the Return key.
- Block indent text by selecting two or more lines and pressing the Tab key.
- Implement or remove SQL style or toggle C style comment notation within your
code.
You can also drag and drop certain objects from the treeview which
can save time in typing long object names. Text containing the object name will be
fully qualified with schema. Double quotes will be added if required.
For functions and procedures, the function name along with parameter names will
be pasted in the Query Tool.
The Data Output Panel
The Data Output panel displays data and statistics generated by the most
recently executed query.
The Data Output tab displays the result set of the query in a table format.
You can:
- Select and copy from the displayed result set.
- Use the Execute/Refresh options to retrieve query execution information and
set query execution options.
- Use the Download as CSV icon to download the content of the Data Output
tab as a comma-delimited file.
- Edit the data in the result set of a SELECT query if it is updatable.
A result set is updatable if:
- All columns are either selected directly from a single table, or
are not table columns at all (e.g. concatenation of 2 columns).
Only columns that are selected directly from the table are
editable, other columns are read-only.
- All the primary key columns or OIDs of the table are selected in the
result set.
Any columns that are renamed or selected more than once are also read-only.
Editable and read-only columns are identified using pencil and lock icons
(respectively) in the column headers.
The psycopg2 driver version should be equal to or above 2.8 for updatable
query result sets to work.
An updatable result set is identical to the Data Grid in
View/Edit Data mode, and can be modified in the same way.
If Auto-commit is off, the data changes are made as part of the ongoing
transaction, if no transaction is ongoing a new one is initiated. The data
changes are not committed to the database unless the transaction is committed.
If any errors occur during saving (for example, trying to save NULL into a
column with NOT NULL constraint) the data changes are rolled back to an
automatically created SAVEPOINT to ensure any previously executed queries in
the ongoing transaction are not rolled back.
All rowsets from previous queries or commands that are displayed in the Data
Output panel will be discarded when you invoke another query; open another
Query Tool tab to keep your previous results available.
Explain Panel
To generate the Explain or Explain Analyze plan of a query, click on
Explain or Explain Analyze button in the toolbar.
More options related to Explain and Explain Analyze can be selected from
the drop down on the right side of Explain Analyze button in the toolbar.
Please note that pgAdmin generates the Explain [Analyze] plan in JSON format.
On successful generation of Explain plan, it will create three tabs/panels
under the Explain panel.
Please note that EXPLAIN VERBOSE cannot be displayed graphically. Hover over
an icon on the Graphical tab to review information about that item; a popup
window will display information about the selected object. For information on
JIT statistics, triggers and a summary, hover over the icon on top-right
corner; a similar popup window will be displayed when appropriate.
Use the download button on top left corner of the Explain canvas to download
the plan as an SVG file.
Note: Download as SVG is not supported on Internet Explorer.
Note that the query plan that accompanies the Explain analyze is available on
the Data Output tab.
Table tab shows the plan details in table format, it generates table format
similar to explain.depsez.com. Each row of the table represent the data for a
Explain Plan Node. It may contain the node information, exclusive timing,
inclusive timing, actual vs planned rows differences, actual rows, planned
rows, loops.
background color of the exclusive, inclusive, and Rows X columns may vary based on the
difference between actual vs planned.
If percentage of the exclusive/inclusive timings of the total query time is:
> 90 - Red color
> 50 - Orange (between red and yellow) color
> 10 - Yellow color
If planner mis-estimated number of rows (actual vs planned) by
10 times - Yellow color
100 times - Orange (between Red and Yellow) color
1000 times - Red color
Statistics tab shows two tables:
1. Statistics per Plan Node Type
2. Statistics per Table
Messages Panel
Use the Messages tab to view information about the most recently executed
query:
If the server returns an error, the error message will be displayed on the
Messages tab, and the syntax that caused the error will be underlined in the
SQL editor. If a query succeeds, the Messages tab displays how long the
query took to complete and how many rows were retrieved:
Query History Panel
Use the Query History tab to review activity for the current session:
The Query History tab displays information about recent commands:
- The date and time that a query was invoked.
- The text of the query.
- The number of rows returned by the query.
- The amount of time it took the server to process the query and return a
result set.
- Messages returned by the server (not noted on the Messages tab).
- The source of the query (indicated by icons corresponding to the toolbar).
You can show or hide the queries generated internally by pgAdmin (during
‘View/Edit Data’ or ‘Save Data’ operations).
To erase the content of the Query History tab, select Clear history from
the Clear drop-down menu.
Query History is maintained across sessions for each database on a per-user
basis when running in Query Tool mode. In View/Edit Data mode, history is not
retained. By default, the last 20 queries are stored for each database. This
can be adjusted in config_local.py by overriding the MAX_QUERY_HIST_STORED
value. See the Deployment section for more information.
Connection Status
Use the Connection status feature to view the current connection and
transaction status by clicking on the status icon in the Query Tool:


Novation ReMOTE SL
Single Patch Editor
This patch editor was originally designed and tested with the ReMOTE SL Mk II keyboards and the Zero MkII. However, it probably works fine with the earlier ReMOTE SL Keyboards and Zero (untested). It may also work properly with the ReMOTE SL Compact (untested), but you have to set it up right (see discussion below). This patch protocol is extremely complex, and had to be entirely reverse-engineered, so the editor is likely to have errors, especially for the untested machines. If you discover irregularities, please contact me (sean@cs.gmu.edu).
This editor was tested on a Zero Mk II with the latest firmware (1.3.14). You'd probably want to update to the latest firmware, but beware that the firmware options available on Novation's website are out of date and incorrect.
Communicating with Edisyn
If you're using USB, set Edisyn's Send and Receive devices to your SL's Port 1 (on my Zero, the device is called "ZeRO MkII Port 1", and make sure that the SL is connecting to at least port U1 when you dump. If you're using 5-Pin DIN MIDI, make sure Edisyn's Send and Receive devices are your MIDI interface, and that the SL is connecting to at least port M1 or M2 as you see fit. Windows/Linux Warning  Java on Windows or Linux does not play nicely with the Novation SL's USB port. Instead, Windows and Linux users should use a good USB MIDI interface.
About the SL Compact
Novation's sysex is very strange in that the same exact sysex is meant to work for the SL Compact and for other SL models, even though they have different capabilities: the same sysex data is reused. The upshot of this is that if you have an SL Compact, you should turn on the "Compact" checkbox. This won't affect your current SL editor (close it), but it will affect future editors.
The Compact version of Edisyn's editor replaces the Faders and Pots with additional banks of Encoders, and eliminates the Cross-fader (which overlaps with a Compact encoder, don't ask). Note that for sysex compatibility some items still exist in the editor which aren't available in the Compact, such as the X1/Y1/X2/Y2 touchpad controls, MIDI 2 options, and the global pot strategy. You can just ignore them.
The Compact does not support MIDI 2. So don't select that.
SL keyboards versus the SL Zero
On the Zero, you have the choice of "Common" vs "Program" Ports and Channels. But on the keyboards, the choice is between "Common" and "Keyboard". Obviously this is because the Zero doesn't have a keyboard. Because I own a Zero, and I think they're more common nowadays, I have elected to use the term "Program" in the editor.
Also, you can set any parameter you like, but note that the keyboards have certain capabilities the Zero does not, and vice versa. Items specific to keyboards are marked with [Key] and the sole item specific to the Zero (Mk II only), namely the Cross-Fader, is marked with [Zero].
Sending and Receiving a Patch
The SL has limited sysex options. Edisyn cannot request a patch (a "template") from the SL -- instead you must manually send Edisyn the patch from the unit.
If you Send to Current Patch from Edisyn, it will upload the patch to the SL without a patch (template) number. You'll then have to save it on the unit directly. If you Write to Patch..., it'll save it directly at the patch number.
Individual parameters cannot be sent. You have to send a whole patch.
You can do a batch download of all the SL's patches, but you cannot specify a range, and because Edisyn cannot request patches from the SL you will have to initiate a bulk dump ("Dump ALL") from your SL after you have asked Edisyn to start the batch download.
About the Buttons
Edisyn groups the buttons into A, B, C, D, and Transport. They are:
- Buttons A (1-8) Top Left
- Buttons B (9-16) Bottom Left
- Buttons C (17-24) Top Right
- Buttons D (25-32) Bottom Right
- Transport Rewind, Fast Forward, Stop, Play, Loop, and Record
On the Compact, these button groups are known as 1, 2, 3, and 4, and Edisyn will call them this as well, so as to be compatible with the Compact's odd nomenclature.
Because it has nowhere else to go, the Cross-Fader is lumped in with the Transport buttons.
About the Layout Category
This category just displays the labels for the controls as your SL would, so it's clear how they're laid out.
Cutting and Pasting
Novation's original editor had considerable flexibility in enabling you to move commands among controls or groups of controls. Edisyn cannot provide this level of flexibility, but it endeavors to give as much as it can. Even though there are some differences between encoders, pots/faders, buttons, drum pads, and pitch bend, you can in fact cut and paste between any control just like in the original editor. This is done by cutting and pasting the controls' categories: just click on the category headers. You can also copy and paste entire tabs, as well as keyboard zones. Edisyn will do its best to match up commands transferred between different kinds of controls.
Bugs in Novation's SLMkII Editor
Novation's editor has a lot of bugs and missing features. Edisyn tries to be compatible with the SL, not the editor. For example:
- The Novation editor sends invalid data regarding ports to the SL which can corrupt its responses later.
- There are serious errors in how the sysex data (DV) position is determined.
- Sysex data (DV) position doesn't work properly for 16-bit encoders -- only one DV is inserted instead of two.
- Novation's editor is missing Roland, MSB-LSB, and LSB-MSB options for sysex.
- Novation's editor allows template (patch) positions to go to 40 when they should only go to 32.
- Novation's editor is missing Realtime, Template, and Bank Change options for buttons. In fact, if the editor reads a patch with a button set to Template, it will incorrectly interpret this as Aftertouch and will misbehave.
- Novation's editor shows default values, but the unit does not (this might be a bug on the unit, it's not clear).
- Novation's editor is missing the crossfader.
Edisyn tries to remedy these errors.
Errors in the Novation SL
The SL also has a few bugs and oddities itself.
- For some reason, you can't edit the default settings of encoders, pots, faders, or the cross-fader directly on the SL. Edisyn lets you edit them.
- The SL permits you to have 14-bit (0-16K) CC parameters with any CC number from 0 to 127, even the MIDI spec only defines 14-bit CC parameters from 0 to 31.
- You'll note a mysterious port called "USB3". This is a hidden port that the SL uses to communicate with Automap software. Don't use it, it's useless now.
- The available bytes for Roland sysex are one less than you'd expect. So if the sysex length is 10, you will have 9 bytes.
Gotchas
- For certain good reasons, you'll find that a few of Edisyn's pop-up menu options are ordered differently than you'd find on the unit or on the original Novation editor.
- This editor can't edit automap templates: but they're dead now anyway.
- I've not found any way to assign useful values to the Automap/Learn/View/User/Fx/Inst/Mixer buttons. Now that Automap is gone, they're probably useless. That's unfortunate.
- There are actually 33 templates. Template #33 is a special default template for Automap, and shouldn't be played with. You will find that the bulk download may include all 33 templates; but Edisyn will only let you write to the first 32.
Hints
- When setting sysex, remember that the editor doesn't display the F0 at the beginning or the F7 at the end (nor the Roland checksum): but the SL does.
- If you need to upload the entire default template settings for some reason (and they're not very good), Edisyn has them in the file NovationSLMKIIDefaults.zip in the resources directory on Edisyn's repository.
- The entire reverse-engineered Novation ReMOTE SL sysex protocol is available at the end of the Java source code for Edisyn's SL editor. Check it out!
Librarian Support
You can't request patches from the SL, thus you can't request banks or ranges of patches -- you have to send them to the librarian manually.
Thanks
Thanks to Focusrite PLC (Novation) for their assistance in gathering certain information helpful in development of this editor. Thank you to Zsolt Szabo<EMAIL_ADDRESS>for bug reports.
| By
| Sean Luke
|
| Date
| February 2021
|


Source
(defmethod deallocate-block (block (heap free-list-heap))
;; Push the block on the front of its free list.
(let* ((size (block-size block heap))
(size-class (size-class size heap)))
(if (free-list-empty-p size-class heap)
;; Let free list start point to the block and vice versa.
(setf (block-header block heap) (free-list-pointer size-class)
(free-list-start heap size-class) block)
;; Normal case: let free list start point to the block,
;; the block to the old block that the free list start pointed to.
(let ((old-first-block (free-list-start heap size-class)))
(setf (block-header block heap) old-first-block
(free-list-start heap size-class) block)))
;;
(initialize-block block size heap)))
Source Context


An application consists of a number of files:
The binary |
This gets installed in /usr/bin . |
A desktop file |
The desktop file provides important information about the application to the desktop shell, such as its name, icon, D-Bus name, commandline to launch it, etc. It is installed in /usr/share/applications . |
An icon |
The icon gets installed in /usr/share/icons/hicolor/48x48/apps , where it will be found regardless of the current theme. |
A settings schema |
If the application uses GSettings, it will install its schema
in /usr/share/glib-2.0/schemas , so that tools
like dconf-editor can find it. |
Other resources |
Other files, such as GtkBuilder ui files, are best loaded from
resources stored in the application binary itself. This eliminates the
need for most of the files that would traditionally be installed in
an application-specific location in /usr/share . |
GTK+ includes application support that is built on top of GApplication. In this tutorial we'll build a simple application by starting from scratch, adding more and more pieces over time. Along the way, we'll learn about GtkApplication, templates, resources, application menus, settings, GtkHeaderBar, GtkStack, GtkSearchBar, GtkListBox, and more.
The full, buildable sources for these examples can be found
in the examples/ directory of the GTK+ source distribution, or
online in the GTK+ git repository.
You can build each example separately by using make with the Makefile.example
file. For more information, see the README
included in the
examples directory.
When using GtkApplication, the main()
function can be very
simple. We just call g_application_run()
and give it an instance
of our application class.
1 2 3 4 5 6 7 8 9 |
#include <gtk/gtk.h> #include "exampleapp.h" int main (int argc, char *argv[]) { return g_application_run (G_APPLICATION (example_app_new ()), argc, argv); } |
All the application logic is in the application class, which is a subclass of GtkApplication. Our example does not yet have any interesting functionality. All it does is open a window when it is activated without arguments, and open the files it is given, if it is started with arguments.
To handle these two cases, we override the activate()
vfunc,
which gets called when the application is launched without commandline
arguments, and the open()
vfunc, which gets called when the application
is launched with commandline arguments.
To learn more about GApplication entry points, consult the GIO documentation.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 |
#include <gtk/gtk.h> #include "exampleapp.h" #include "exampleappwin.h" struct _ExampleApp { GtkApplication parent; }; G_DEFINE_TYPE(ExampleApp, example_app, GTK_TYPE_APPLICATION); static void example_app_init (ExampleApp *app) { } static void example_app_activate (GApplication *app) { ExampleAppWindow *win; win = example_app_window_new (EXAMPLE_APP (app)); gtk_window_present (GTK_WINDOW (win)); } static void example_app_open (GApplication *app, GFile **files, gint n_files, const gchar *hint) { GList *windows; ExampleAppWindow *win; int i; windows = gtk_application_get_windows (GTK_APPLICATION (app)); if (windows) win = EXAMPLE_APP_WINDOW (windows->data); else win = example_app_window_new (EXAMPLE_APP (app)); for (i = 0; i < n_files; i++) example_app_window_open (win, files[i]); gtk_window_present (GTK_WINDOW (win)); } static void example_app_class_init (ExampleAppClass *class) { G_APPLICATION_CLASS (class)->activate = example_app_activate; G_APPLICATION_CLASS (class)->open = example_app_open; } ExampleApp * example_app_new (void) { return g_object_new (EXAMPLE_APP_TYPE, "application-id", "org.gtk.exampleapp", "flags", G_APPLICATION_HANDLES_OPEN, NULL); } |
Another important class that is part of the application support in GTK+ is GtkApplicationWindow. It is typically subclassed as well. Our subclass does not do anything yet, so we will just get an empty window.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
#include <gtk/gtk.h> #include "exampleapp.h" #include "exampleappwin.h" struct _ExampleAppWindow { GtkApplicationWindow parent; }; G_DEFINE_TYPE(ExampleAppWindow, example_app_window, GTK_TYPE_APPLICATION_WINDOW); static void example_app_window_init (ExampleAppWindow *app) { } static void example_app_window_class_init (ExampleAppWindowClass *class) { } ExampleAppWindow * example_app_window_new (ExampleApp *app) { return g_object_new (EXAMPLE_APP_WINDOW_TYPE, "application", app, NULL); } void example_app_window_open (ExampleAppWindow *win, GFile *file) { } |
As part of the initial setup of our application, we also create an icon and a desktop file.
1 2 3 4 5 6 |
[Desktop Entry] Type=Application Name=Example Icon=exampleapp StartupNotify=true Exec=@bindir@/exampleapp |
Note that @bindir@
needs to be replaced
with the actual path to the binary before this desktop file can be used.
Here is what we've achieved so far:
This does not look very impressive yet, but our application is already presenting itself on the session bus, it has single-instance semantics, and it accepts files as commandline arguments.
In this step, we use a GtkBuilder template to associate a GtkBuilder ui file with our application window class.
Our simple ui file puts a GtkHeaderBar on top of a GtkStack widget. The header bar contains a GtkStackSwitcher, which is a standalone widget to show a row of 'tabs' for the pages of a GtkStack.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
<?xml version="1.0" encoding="UTF-8"?> <interface>  <template class="ExampleAppWindow" parent="GtkApplicationWindow"> <property name="title" translatable="yes">Example Application</property> <property name="default-width">600</property> <property name="default-height">400</property> <child> <object class="GtkBox" id="content_box"> <property name="visible">True</property> <property name="orientation">vertical</property> <child> <object class="GtkHeaderBar" id="header"> <property name="visible">True</property> <child type="title"> <object class="GtkStackSwitcher" id="tabs"> <property name="visible">True</property> <property name="stack">stack</property> </object> </child> </object> </child> <child> <object class="GtkStack" id="stack"> <property name="visible">True</property> </object> </child> </object> </child> </template> </interface> |
To make use of this file in our application, we revisit
our GtkApplicationWindow subclass, and call
gtk_widget_class_set_template_from_resource()
from the class init
function to set the ui file as template for this class. We also
add a call to gtk_widget_init_template()
in the instance init
function to instantiate the template for each instance of our
class.
... static void example_app_window_init (ExampleAppWindow *win) { gtk_widget_init_template (GTK_WIDGET (win)); } static void example_app_window_class_init (ExampleAppWindowClass *class) { gtk_widget_class_set_template_from_resource (GTK_WIDGET_CLASS (class), "/org/gtk/exampleapp/window.ui"); } ...
You may have noticed that we used the _from_resource()
variant
of the function that sets a template. Now we need to use GLib's resource
functionality to include the ui file in the binary. This is commonly
done by listing all resources in a .gresource.xml file, such as this:
1 2 3 4 5 6 |
<?xml version="1.0" encoding="UTF-8"?> <gresources> <gresource prefix="/org/gtk/exampleapp"> <file preprocess="xml-stripblanks">window.ui</file> </gresource> </gresources> |
This file has to be converted into a C source file that will be compiled and linked into the application together with the other source files. To do so, we use the glib-compile-resources utility:
glib-compile-resources exampleapp.gresource.xml --target=resources.c --generate-source
Our application now looks like this:
In this step, we make our application show the content of all the files that it is given on the commandline.
To this end, we add a private struct to our application
window subclass and keep a reference to the GtkStack there.
The gtk_widget_class_bind_template_child_private()
function
arranges things so that after instantiating the template, the
stack
member of the private struct will point to the widget of
the same name from the template.
... struct _ExampleAppWindowPrivate { GtkWidget *stack; }; G_DEFINE_TYPE_WITH_PRIVATE(ExampleAppWindow, example_app_window, GTK_TYPE_APPLICATION_WINDOW); ... static void example_app_window_class_init (ExampleAppWindowClass *class) { gtk_widget_class_set_template_from_resource (GTK_WIDGET_CLASS (class), "/org/gtk/exampleapp/window.ui"); gtk_widget_class_bind_template_child_private (GTK_WIDGET_CLASS (class), ExampleAppWindow, stack); } ...
Now we revisit the example_app_window_open()
function that
is called for each commandline argument, and construct a GtkTextView
that we then add as a page to the stack:
... void example_app_window_open (ExampleAppWindow *win, GFile *file) { ExampleAppWindowPrivate *priv; gchar *basename; GtkWidget *scrolled, *view; gchar *contents; gsize length; priv = example_app_window_get_instance_private (win); basename = g_file_get_basename (file); scrolled = gtk_scrolled_window_new (NULL, NULL); gtk_widget_show (scrolled); gtk_widget_set_hexpand (scrolled, TRUE); gtk_widget_set_vexpand (scrolled, TRUE); view = gtk_text_view_new (); gtk_text_view_set_editable (GTK_TEXT_VIEW (view), FALSE); gtk_text_view_set_cursor_visible (GTK_TEXT_VIEW (view), FALSE); gtk_widget_show (view); gtk_container_add (GTK_CONTAINER (scrolled), view); gtk_stack_add_titled (GTK_STACK (priv->stack), scrolled, basename, basename); if (g_file_load_contents (file, NULL, &contents, &length, NULL, NULL)) { GtkTextBuffer *buffer; buffer = gtk_text_view_get_buffer (GTK_TEXT_VIEW (view)); gtk_text_buffer_set_text (buffer, contents, length); g_free (contents); } g_free (basename); } ...
Note that we did not have to touch the stack switcher
at all. It gets all its information from the stack that it
belongs to. Here, we are passing the label to show for each
file as the last argument to the gtk_stack_add_titled()
function.
Our application is beginning to take shape:
An application menu is shown by GNOME shell at the top of the screen. It is meant to collect infrequently used actions that affect the whole application.
Just like the window template, we specify our application menu in a ui file, and add it as a resource to our binary.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
<?xml version="1.0"?> <interface>  <menu id="appmenu">
 <item> <attribute name="label" translatable="yes">_Preferences</attribute> <attribute name="action">app.preferences</attribute> </item>

 <item> <attribute name="label" translatable="yes">_Quit</attribute> <attribute name="action">app.quit</attribute> </item>
 </menu> </interface> |
To associate the app menu with the application, we have to call
gtk_application_set_app_menu()
. Since app menus work by activating
GActions, we also have to add a suitable set of actions to our
application.
Both of these tasks are best done in the startup()
vfunc,
which is guaranteed to be called once for each primary application
instance:
... static void preferences_activated (GSimpleAction *action, GVariant *parameter, gpointer app) { } static void quit_activated (GSimpleAction *action, GVariant *parameter, gpointer app) { g_application_quit (G_APPLICATION (app)); } static GActionEntry app_entries[] = { { "preferences", preferences_activated, NULL, NULL, NULL }, { "quit", quit_activated, NULL, NULL, NULL } }; static void example_app_startup (GApplication *app) { GtkBuilder *builder; GMenuModel *app_menu; const gchar *quit_accels[2] = { "<Ctrl>Q", NULL }; G_APPLICATION_CLASS (example_app_parent_class)->startup (app); g_action_map_add_action_entries (G_ACTION_MAP (app), app_entries, G_N_ELEMENTS (app_entries), app); gtk_application_set_accels_for_action (GTK_APPLICATION (app), "app.quit", quit_accels); builder = gtk_builder_new_from_resource ("/org/gtk/exampleapp/app-menu.ui"); app_menu = G_MENU_MODEL (gtk_builder_get_object (builder, "appmenu")); gtk_application_set_app_menu (GTK_APPLICATION (app), app_menu); g_object_unref (builder); } static void example_app_class_init (ExampleAppClass *class) { G_APPLICATION_CLASS (class)->startup = example_app_startup; ... } ...
Our preferences menu item does not do anything yet,
but the Quit menu item is fully functional. Note that it
can also be activated by the usual Ctrl-Q shortcut. The
shortcut was added with gtk_application_set_accels_for_action()
.
The application menu looks like this:
A typical application will have a some preferences that should be remembered from one run to the next. Even for our simple example application, we may want to change the font that is used for the content.
We are going to use GSettings to store our preferences. GSettings requires a schema that describes our settings:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
<?xml version="1.0" encoding="UTF-8"?> <schemalist> <schema path="/org/gtk/exampleapp/" id="org.gtk.exampleapp"> <key name="font" type="s"> <default>'Monospace 12'</default> Font <description>The font to be used for content.</description> </key> <key name="transition" type="s"> <choices> <choice value='none'/> <choice value='crossfade'/> <choice value='slide-left-right'/> </choices> <default>'none'</default> Transition <description>The transition to use when switching tabs.</description> </key> </schema> </schemalist> |
Before we can make use of this schema in our application, we need to compile it into the binary form that GSettings expects. GIO provides macros to do this in autotools-based projects.
Next, we need to connect our settings to the widgets that they are supposed to control. One convenient way to do this is to use GSettings bind functionality to bind settings keys to object properties, as we do here for the transition setting.
... static void example_app_window_init (ExampleAppWindow *win) { ExampleAppWindowPrivate *priv; priv = example_app_window_get_instance_private (win); gtk_widget_init_template (GTK_WIDGET (win)); priv->settings = g_settings_new ("org.gtk.exampleapp"); g_settings_bind (priv->settings, "transition", priv->stack, "transition-type", G_SETTINGS_BIND_DEFAULT); } ...
The code to connect the font setting is a little more involved, since there is no simple object property that it corresponds to, so we are not going to go into that here.
At this point, the application will already react if you change one of the settings, e.g. using the gsettings commandline tool. Of course, we expect the application to provide a preference dialog for these. So lets do that now. Our preference dialog will be a subclass of GtkDialog, and we'll use the same techniques that we've already seen: templates, private structs, settings bindings.
Lets start with the template.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 |
<?xml version="1.0" encoding="UTF-8"?> <interface>  <template class="ExampleAppPrefs" parent="GtkDialog"> <property name="title" translatable="yes">Preferences</property> <property name="resizable">False</property> <property name="modal">True</property> <child internal-child="vbox"> <object class="GtkBox" id="vbox"> <child> <object class="GtkGrid" id="grid"> <property name="visible">True</property> <property name="margin">6</property> <property name="row-spacing">12</property> <property name="column-spacing">6</property> <child> <object class="GtkLabel" id="fontlabel"> <property name="visible">True</property> <property name="label">_Font:</property> <property name="use-underline">True</property> <property name="mnemonic-widget">font</property> <property name="xalign">1</property> </object> <packing> <property name="left-attach">0</property> <property name="top-attach">0</property> </packing> </child> <child> <object class="GtkFontButton" id="font"> <property name="visible">True</property> </object> <packing> <property name="left-attach">1</property> <property name="top-attach">0</property> </packing> </child> <child> <object class="GtkLabel" id="transitionlabel"> <property name="visible">True</property> <property name="label">_Transition:</property> <property name="use-underline">True</property> <property name="mnemonic-widget">transition</property> <property name="xalign">1</property> </object> <packing> <property name="left-attach">0</property> <property name="top-attach">1</property> </packing> </child> <child> <object class="GtkComboBoxText" id="transition"> <property name="visible">True</property> <items> <item translatable="yes" id="none">None</item> <item translatable="yes" id="crossfade">Fade</item> <item translatable="yes" id="slide-left-right">Slide</item> </items> </object> <packing> <property name="left-attach">1</property> <property name="top-attach">1</property> </packing> </child> </object> </child> </object> </child> </template> </interface> |
Next comes the dialog subclass.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 |
#include <gtk/gtk.h> #include "exampleapp.h" #include "exampleappwin.h" #include "exampleappprefs.h" struct _ExampleAppPrefs { GtkDialog parent; }; typedef struct _ExampleAppPrefsPrivate ExampleAppPrefsPrivate; struct _ExampleAppPrefsPrivate { GSettings *settings; GtkWidget *font; GtkWidget *transition; }; G_DEFINE_TYPE_WITH_PRIVATE(ExampleAppPrefs, example_app_prefs, GTK_TYPE_DIALOG) static void example_app_prefs_init (ExampleAppPrefs *prefs) { ExampleAppPrefsPrivate *priv; priv = example_app_prefs_get_instance_private (prefs); gtk_widget_init_template (GTK_WIDGET (prefs)); priv->settings = g_settings_new ("org.gtk.exampleapp"); g_settings_bind (priv->settings, "font", priv->font, "font", G_SETTINGS_BIND_DEFAULT); g_settings_bind (priv->settings, "transition", priv->transition, "active-id", G_SETTINGS_BIND_DEFAULT); } static void example_app_prefs_dispose (GObject *object) { ExampleAppPrefsPrivate *priv; priv = example_app_prefs_get_instance_private (EXAMPLE_APP_PREFS (object)); g_clear_object (&priv->settings); G_OBJECT_CLASS (example_app_prefs_parent_class)->dispose (object); } static void example_app_prefs_class_init (ExampleAppPrefsClass *class) { G_OBJECT_CLASS (class)->dispose = example_app_prefs_dispose; gtk_widget_class_set_template_from_resource (GTK_WIDGET_CLASS (class), "/org/gtk/exampleapp/prefs.ui"); gtk_widget_class_bind_template_child_private (GTK_WIDGET_CLASS (class), ExampleAppPrefs, font); gtk_widget_class_bind_template_child_private (GTK_WIDGET_CLASS (class), ExampleAppPrefs, transition); } ExampleAppPrefs * example_app_prefs_new (ExampleAppWindow *win) { return g_object_new (EXAMPLE_APP_PREFS_TYPE, "transient-for", win, "use-header-bar", TRUE, NULL); } |
Now we revisit the preferences_activated()
function in our
application class, and make it open a new preference dialog.
... static void preferences_activated (GSimpleAction *action, GVariant *parameter, gpointer app) { ExampleAppPrefs *prefs; GtkWindow *win; win = gtk_application_get_active_window (GTK_APPLICATION (app)); prefs = example_app_prefs_new (EXAMPLE_APP_WINDOW (win)); gtk_window_present (GTK_WINDOW (prefs)); } ...
After all this work, our application can now show a preference dialog like this:
We continue to flesh out the functionality of our application. For now, we add search. GTK+ supports this with GtkSearchEntry and GtkSearchBar. The search bar is a widget that can slide in from the top to present a search entry.
We add a toggle button to the header bar, which can be used to slide out the search bar below the header bar.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 |
<?xml version="1.0" encoding="UTF-8"?> <interface>  <template class="ExampleAppWindow" parent="GtkApplicationWindow"> <property name="title" translatable="yes">Example Application</property> <property name="default-width">600</property> <property name="default-height">400</property> <child> <object class="GtkBox" id="content_box"> <property name="visible">True</property> <property name="orientation">vertical</property> <child> <object class="GtkHeaderBar" id="header"> <property name="visible">True</property> <child type="title"> <object class="GtkStackSwitcher" id="tabs"> <property name="visible">True</property> <property name="stack">stack</property> </object> </child> <child> <object class="GtkToggleButton" id="search"> <property name="visible">True</property> <property name="sensitive">False</property>  <child> <object class="GtkImage" id="search-icon"> <property name="visible">True</property> <property name="icon-name">edit-find-symbolic</property> <property name="icon-size">1</property> </object> </child> </object> <packing> <property name="pack-type">end</property> </packing> </child> </object> </child> <child> <object class="GtkSearchBar" id="searchbar"> <property name="visible">True</property> <child> <object class="GtkSearchEntry" id="searchentry"> <signal name="search-changed" handler="search_text_changed"/> <property name="visible">True</property> </object> </child> </object> </child> <child> <object class="GtkStack" id="stack"> <signal name="notify::visible-child" handler="visible_child_changed"/> <property name="visible">True</property> </object> </child> </object> </child> </template> </interface> |
Implementing the search needs quite a few code changes that we are not going to completely go over here. The central piece of the search implementation is a signal handler that listens for text changes in the search entry.
... static void search_text_changed (GtkEntry *entry, ExampleAppWindow *win) { ExampleAppWindowPrivate *priv; const gchar *text; GtkWidget *tab; GtkWidget *view; GtkTextBuffer *buffer; GtkTextIter start, match_start, match_end; text = gtk_entry_get_text (entry); if (text[0] == '\0') return; priv = example_app_window_get_instance_private (win); tab = gtk_stack_get_visible_child (GTK_STACK (priv->stack)); view = gtk_bin_get_child (GTK_BIN (tab)); buffer = gtk_text_view_get_buffer (GTK_TEXT_VIEW (view)); /* Very simple-minded search implementation */ gtk_text_buffer_get_start_iter (buffer, &start); if (gtk_text_iter_forward_search (&start, text, GTK_TEXT_SEARCH_CASE_INSENSITIVE, &match_start, &match_end, NULL)) { gtk_text_buffer_select_range (buffer, &match_start, &match_end); gtk_text_view_scroll_to_iter (GTK_TEXT_VIEW (view), &match_start, 0.0, FALSE, 0.0, 0.0); } } static void example_app_window_init (ExampleAppWindow *win) { ... gtk_widget_class_bind_template_callback (GTK_WIDGET_CLASS (class), search_text_changed); ... } ...
With the search bar, our application now looks like this:
As another piece of functionality, we are adding a sidebar, which demonstrates GtkMenuButton, GtkRevealer and GtkListBox.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 |
<?xml version="1.0" encoding="UTF-8"?> <interface>  <template class="ExampleAppWindow" parent="GtkApplicationWindow"> <property name="title" translatable="yes">Example Application</property> <property name="default-width">600</property> <property name="default-height">400</property> <child> <object class="GtkBox" id="content_box"> <property name="visible">True</property> <property name="orientation">vertical</property> <child> <object class="GtkHeaderBar" id="header"> <property name="visible">True</property> <child type="title"> <object class="GtkStackSwitcher" id="tabs"> <property name="visible">True</property> <property name="stack">stack</property> </object> </child> <child> <object class="GtkToggleButton" id="search"> <property name="visible">True</property> <property name="sensitive">False</property>  <child> <object class="GtkImage" id="search-icon"> <property name="visible">True</property> <property name="icon-name">edit-find-symbolic</property> <property name="icon-size">1</property> </object> </child> </object> <packing> <property name="pack-type">end</property> </packing> </child> <child> <object class="GtkMenuButton" id="gears"> <property name="visible">True</property> <property name="direction">none</property> <property name="use-popover">True</property>  </object> <packing> <property name="pack-type">end</property> </packing> </child> </object> </child> <child> <object class="GtkSearchBar" id="searchbar"> <property name="visible">True</property> <child> <object class="GtkSearchEntry" id="searchentry"> <signal name="search-changed" handler="search_text_changed"/> <property name="visible">True</property> </object> </child> </object> </child> <child> <object class="GtkBox" id="hbox"> <property name="visible">True</property> <child> <object class="GtkRevealer" id="sidebar"> <property name="visible">True</property> <property name="transition-type">slide-right</property> <child> <object class="GtkScrolledWindow" id="sidebar-sw"> <property name="visible">True</property> <property name="hscrollbar-policy">never</property> <property name="vscrollbar-policy">automatic</property> <child> <object class="GtkListBox" id="words"> <property name="visible">True</property> <property name="selection-mode">none</property> </object> </child> </object> </child> </object> </child> <child> <object class="GtkStack" id="stack"> <signal name="notify::visible-child" handler="visible_child_changed"/> <property name="visible">True</property> </object> </child> </object> </child> </object> </child> </template> </interface> |
The code to populate the sidebar with buttons for the words found in each file is a little too involved to go into here. But we'll look at the code to add the gears menu.
As expected by now, the gears menu is specified in a GtkBuilder ui file.
1 2 3 4 5 6 7 8 9 10 11 12 |
<?xml version="1.0"?> <interface>  <menu id="menu">
 <item> <attribute name="label" translatable="yes">_Words</attribute> <attribute name="action">win.show-words</attribute> </item>
 </menu> </interface> |
To connect the menuitem to the show-words setting, we use a GAction corresponding to the given GSettings key.
... static void example_app_window_init (ExampleAppWindow *win) { ... builder = gtk_builder_new_from_resource ("/org/gtk/exampleapp/gears-menu.ui"); menu = G_MENU_MODEL (gtk_builder_get_object (builder, "menu")); gtk_menu_button_set_menu_model (GTK_MENU_BUTTON (priv->gears), menu); g_object_unref (builder); action = g_settings_create_action (priv->settings, "show-words"); g_action_map_add_action (G_ACTION_MAP (win), action); g_object_unref (action); } ...
What our application looks like now:
Widgets and other objects have many useful properties.
Here we show some ways to use them in new and flexible ways, by wrapping them in actions with GPropertyAction or by binding them with GBinding.
To set this up, we add two labels to the header bar in our
window template, named lines_label
and lines
, and bind them to
struct members in the private struct, as we've seen a couple of times
by now.
We add a new "Lines" menu item to the gears menu, which triggers the show-lines action:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
<?xml version="1.0"?> <interface>  <menu id="menu">
 <item> <attribute name="label" translatable="yes">_Words</attribute> <attribute name="action">win.show-words</attribute> </item> <item> <attribute name="label" translatable="yes">_Lines</attribute> <attribute name="action">win.show-lines</attribute> </item>
 </menu> </interface> |
To make this menu item do something, we create a property
action for the visible property of the lines
label, and add it to the
actions of the window. The effect of this is that the visibility
of the label gets toggled every time the action is activated.
Since we want both labels to appear and disappear together,
we bind the visible property of the lines_label
widget to the
same property of the lines
widget.
... static void example_app_window_init (ExampleAppWindow *win) { ... action = (GAction*) g_property_action_new ("show-lines", priv->lines, "visible"); g_action_map_add_action (G_ACTION_MAP (win), action); g_object_unref (action); g_object_bind_property (priv->lines, "visible", priv->lines_label, "visible", G_BINDING_DEFAULT); } ...
We also need a function that counts the lines of the currently
active tab, and updates the lines
label. See the
full source
if you are interested in the details.
This brings our example application to this appearance:
Our application already uses a GtkHeaderBar, but so far it still gets a 'normal' window titlebar on top of that. This is a bit redundant, and we will now tell GTK+ to use the header bar as replacement for the titlebar. To do so, we move it around to be a direct child of the window, and set its type to be titlebar.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 |
<?xml version="1.0" encoding="UTF-8"?> <interface>  <template class="ExampleAppWindow" parent="GtkApplicationWindow"> <property name="title" translatable="yes">Example Application</property> <property name="default-width">600</property> <property name="default-height">400</property> <child type="titlebar"> <object class="GtkHeaderBar" id="header"> <property name="visible">True</property> <property name="show-close-button">True</property> <child> <object class="GtkLabel" id="lines_label"> <property name="visible">False</property> <property name="label" translatable="yes">Lines:</property> </object> <packing> <property name="pack-type">start</property> </packing> </child> <child> <object class="GtkLabel" id="lines"> <property name="visible">False</property> </object> <packing> <property name="pack-type">start</property> </packing> </child> <child type="title"> <object class="GtkStackSwitcher" id="tabs"> <property name="visible">True</property> <property name="stack">stack</property> </object> </child> <child> <object class="GtkToggleButton" id="search"> <property name="visible">True</property> <property name="sensitive">False</property>  <child> <object class="GtkImage" id="search-icon"> <property name="visible">True</property> <property name="icon-name">edit-find-symbolic</property> <property name="icon-size">1</property> </object> </child> </object> <packing> <property name="pack-type">end</property> </packing> </child> <child> <object class="GtkMenuButton" id="gears"> <property name="visible">True</property> <property name="direction">none</property> <property name="use-popover">True</property>  </object> <packing> <property name="pack-type">end</property> </packing> </child> </object> </child> <child> <object class="GtkBox" id="content_box"> <property name="visible">True</property> <property name="orientation">vertical</property> <child> <object class="GtkSearchBar" id="searchbar"> <property name="visible">True</property> <child> <object class="GtkSearchEntry" id="searchentry"> <signal name="search-changed" handler="search_text_changed"/> <property name="visible">True</property> </object> </child> </object> </child> <child> <object class="GtkBox" id="hbox"> <property name="visible">True</property> <child> <object class="GtkRevealer" id="sidebar"> <property name="visible">True</property> <property name="transition-type">slide-right</property> <child> <object class="GtkScrolledWindow" id="sidebar-sw"> <property name="visible">True</property> <property name="hscrollbar-policy">never</property> <property name="vscrollbar-policy">automatic</property> <child> <object class="GtkListBox" id="words"> <property name="visible">True</property> <property name="selection-mode">none</property> </object> </child> </object> </child> </object> </child> <child> <object class="GtkStack" id="stack"> <signal name="notify::visible-child" handler="visible_child_changed"/> <property name="visible">True</property> </object> </child> </object> </child> </object> </child> </template> </interface> |
A small extra bonus of using a header bar is that we get a fallback application menu for free. Here is how the application now looks, if this fallback is used.
If we set up the window icon for our window, the menu button will use that instead of the generic placeholder icon you see here.


Class SPI – Serial Peripheral Interface bus control (master)¶
SPI is a synchronous serial protocol that is driven by a master.
At the physical level, a bus consists of 3 lines, SCK, MOSI and MISO.
Multiple devices can share the same bus. Each device should have
a separate 4th signal, SS (Slave Select), to select a particular
device on a bus with which communication takes place. Management of
an SS signal should happen in user code (with the machine.Pin
class).
This class only support SPI transactions as master. It also supports a software implementation (bitbanging) implementation of SPI by using a negative ID.
When using positive IDs note that the id corresponds to a
physical USART peripheral on the device. This means
that using SPI(0)
causes UART(0)
to be unavailable.
On the Thunderboard Sense, SPI(0)
is used for printing the REPL.
Configuring the use of this peripheral as a SPI peripheral looses all
printing functionality as well as the command line.
SPI(1)
is already initialized as a SPI-bus with the following pins:
MOSI=PC6, MISO=PC7 and SCK=PC8. It is initialized to a baudrate of 8000000.
SPI(1)
can be utilized as is. If the configuration is changed,
the flash as well as the IMU will be unavailable. The bus can be reset to the
initial state with the function
tbsense.reinit_peripherals()
.
Example:
from machine import SPI, Pin
#Initialize peripheral and SS-pin
spi = SPI(1, sck="PC8", mosi="PC6", miso="PC7")
ss_pin = Pin("PC9", Pin.OUT, Pin.PULL_NONE) #Slave associated with pin PC9
#Prepare transaction data
buf = bytearray("EFM32")
#Perform transaction
ss_pin(1)
spi.write_readinto(buf, buf)
ss_pin(0)
print("Received: ", buf)
Constructors¶
-
class
machine.
SPI
(id, ...)¶ Construct an SPI object on the peripheral associated with
id
. The values ofid
are integers from 0 and up for a hardware SPI peripheral. The number of available hardware SPI peripherals depend on the underlying hardware. A negative value gives a software (bitbanging) implementation of SPI.With no additional parameters, the SPI object is created, but not initialized (it retains the settings from last initialization). If any more arguments are given, the bus is initialized. See
init
for the parameters for initialization.
Methods¶
-
SPI.
init
(baudrate=1000000, *, polarity=0, phase=0, bits=8, firstbit=SPI.MSB, sck=None, mosi=None, miso=None)¶ Initialize the SPI peripheral with the given parameters:
baudrate
is the SCK clock rate.`polarity
can be either 0 or 1 and is the level the idle clock line sits at.phase
can be either 0 or 1 to sample data on the first or second clock edge respectively.bits
is the width in bits of each transfer. It can be an arbitrary size between 4 and 16 bits.firstbit
decides if the most or the least significant bit is sent first. Can be eitherSPI.MSB
orSPI.LSB
.sck
,mosi
,miso
are the pin id or objects (seemachine.Pin
).
-
SPI.
deinit
()¶ Turn off the SPI bus.
-
SPI.
read
(nbytes, write=0x00)¶ Read a number of bytes specified by
nbytes
while continously writing the single byte given bywrite
. Returns abytes
object with the data that was read.
-
SPI.
readinto
(buf, write=0x00)¶ Read into the buffer specified by
buf
while continously writing the single byte given by`write
. This function is safe from an interrupt context. ReturnsNone
.
-
SPI.
write
(buf)¶ Write the bytes contained in
buf
. Ifbits
during initialization is larger than 8, the word is sent as [bits
- 8 LSB of byte[2*n+1], byte[2*n]]. ReturnsNone
.
-
SPI.
write_readinto
(write_buf, read_buf)¶ Write the bytes from
write_buf
while reading intoread_buf
. The buffers can be the same or different, but both buffers must have the same length. ReturnsNone
.


{% load i18n %}
{% trans "There's been an error. It's been reported to the site administrators via email and should be fixed shortly. Thanks for your patience." %}


PyTorch Mobile provides a runtime environment to execute state-of-the-art machine learning models on mobile devices. Latency is reduced, privacy preserved, and models can run on mobile devices anytime, anywhere.
In this blog post, we provide a quick overview of 10 currently available PyTorch Mobile powered demo apps running various state-of-the-art PyTorch 1.9 machine learning models spanning images, video, audio and text.
It’s never been easier to deploy a state-of-the-art ML model to a phone. You don’t need any domain knowledge in Machine Learning and we hope one of the below examples resonates enough with you to be the starting point for your next project.
Computer Vision
Image Classification
This app demonstrates how to use PyTorch C++ libraries on iOS and Android to classify a static image with the MobileNetv2/3 model.
iOS #1 iOS #2 Android #1 Android #2
Live Image Classification
This app demonstrates how to run a quantized MobileNetV2 and Resnet18 models to classify images in real time with an iOS and Android device camera.
Image Segmentation
This app demonstrates how to use the PyTorch DeepLabV3 model to segment images. The updated app for PyTorch 1.9 also demonstrates how to create the model using the Mobile Interpreter and load the model with the LiteModuleLoader API.
Vision Transformer for Handwritten Digit Recognition
This app demonstrates how to use Facebook’s latest optimized Vision Transformer DeiT model to do image classification and handwritten digit recognition.
Object Detection
This app demonstrates how to convert the popular YOLOv5 model and use it on an iOS app that detects objects from pictures in your photos, taken with camera, or with live camera.
D2Go
This app demonstrates how to create and use a much lighter and faster Facebook D2Go model to detect objects from pictures in your photos, taken with camera, or with live camera.
Video
Video Classification
This app demonstrates how to use a pre-trained PyTorchVideo model to perform video classification on tested videos, videos from the Photos library, or even real-time videos.
Natural Language Processing
Text Classification
This app demonstrates how to use a pre-trained Reddit model to perform text classification.
Machine Translation
This app demonstrates how to convert a sequence-to-sequence neural machine translation model trained with the code in the PyTorch NMT tutorial for french to english translation.
Question Answering
This app demonstrates how to use the DistilBERT Hugging Face transformer model to answer questions about Pytorch Mobile itself.
Audio
Speech Recognition
This app demonstrates how to convert Facebook AI’s torchaudio-powered wav2vec 2.0, one of the leading models in speech recognition to TorchScript before deploying it.
We really hope one of these demo apps stood out for you. For the full list, make sure to visit the iOS and Android demo app repos. You should also definitely check out the video An Overview of the PyTorch Mobile Demo Apps which provides both an overview of the PyTorch mobile demo apps and a deep dive into the PyTorch Video app for iOS and Android.


Address:31 A. Mabini Street, Baguio City
Mobile No.: 0927 339 9262
Email<EMAIL_ADDRESS>Age: 19
Date of Birth: March 30, 1997
Nationality: Filipino
Gender: Male
Field of Study: Computer Science
Major: Digital Arts and Animation Track
University: University of the Cordilleras