Reference manual
This page lists functions and methods available in Meson scripts. For more in-depth documentation on how to use them, refer to the manual. For more basic information about how Meson works, take a look at the syntax reference.
Functions
The following functions are available in build files. Click on each to see the description and usage. The objects returned by them are list afterwards.
- add_global_arguments
- add_global_link_arguments
- add_languages
- add_project_arguments
- add_project_link_arguments
- add_test_setup
- benchmark
- build_target
- configuration_data
- configure_file
- custom_target
- declare_dependency
- dependency
- error
- environment
- executable
- find_program
- find_library
- files
- generator
- get_option
- get_variable
- import
- include_directories
- install_data
- install_headers
- install_man
- install_subdir
- is_variable
- jar
- join_paths
- library
- message
- project
- run_command
- run_target
- set_variable
- shared_library
- shared_module
- static_library
- subdir
- subproject
- test
- vcs_tag
Built-in objects
These are built-in objects that are always available.
Returned objects
These are objects returned by the functions listed above.
- build target object
- compiler object
- run result object
- configuration data object
- dependency object
- external program object
- environment object
- external library object
- generator object
- subproject object
add_global_arguments
void add_global_arguments(arg1, arg2, ...)
Adds the positional arguments to the compiler command line for the language specified in language
keyword argument. Note that there is no way to remove an argument set in this way. If you have an argument that is only used in a subset of targets, you have to specify it in per-target flags.
The arguments are used in all compiler invocations with the exception of compile tests, because you might need to run a compile test with and without the argument in question. For this reason only the arguments explicitly specified are used during compile tests.
Note: Usually you should use add_project_arguments
instead, because that works even when you project is used as a subproject.
Note: You must pass always arguments individually arg1, arg2, ...
rather than as a string 'arg1 arg2', ...
add_global_link_arguments
void add_global_link_arguments(*arg1*, *arg2*, ...)
Like add_global_arguments
but the arguments are passed to the linker.
add_languages
add_languages(*langs*)
Add support for new programming languages. Equivalent to having them in the project
declaration. This function is usually used to add languages that are only used on some platforms like this:
project('foobar', 'c')
if compiling_for_osx
add_languages('objc')
endif
Takes one keyword argument, required
. It defaults to true
, which means that if any of the languages specified is not found, Meson will halt. Returns true if all languages specified were found and false otherwise.
add_project_arguments
void add_project_arguments(arg1, arg2, ...)
This function behaves in the same way as add_global_arguments
except that the arguments are only used for the current project, they won't be used in any other subproject.
add_project_link_arguments
void add_project_link_arguments(*arg1*, *arg2*, ...)
Like add_project_arguments
but the arguments are passed to the linker.
add_test_setup
void add_test_setup(*name*, ...)
Add a custom test setup that can be used to run the tests with a custom setup, for example under Valgrind. The keyword arguments are the following:
exe_wrapper
a list containing the wrapper command or script followed by the arguments to itgdb
iftrue
, the tests are also run undergdb
timeout_multiplier
a number to multiply the test timeout withenv
an environment object to use a custom environment
To use the test setup, run mesontest --setup=*name*
inside the build dir.
Note that all these options are also available while running the mesontest
script for running tests instead of ninja test
or msbuild RUN_TESTS.vcxproj
, etc depending on the backend.
benchmark
void benchmark(name, executable, ...)
Creates a benchmark item that will be run when the benchmark target is run. The behaviour of this function is identical to test
with the exception that there is no is_parallel
keyword, because benchmarks are never run in parallel.
build_target
Creates a build target whose type can be set dynamically with the target_type
keyword argument. This declaration:
executable(<arguments and keyword arguments>)
is equivalent to this:
build_target(<arguments and keyword arguments>, target_type : 'executable')
The object returned by build_target
and all convenience wrappers for build_target
such as executable
and library
has methods that are documented in the object methods section below.
configuration_data
configuration_data_object = configuration_data()
Creates an empty configuration object. You should add your configuration with its method calls and finally use it in a call to configure_file
.
configure_file
generated_file = configure_file(...)
This function can run in two modes depending on the keyword arguments passed to it.
When a configuration_data()
object is passed to the configuration:
keyword argument, it takes a template file as the input:
(optional) and produces the output:
(required) by substituting values from the configuration data as detailed in the configuration file documentation.
When a list of strings is passed to the command:
keyword argument, it takes any source or configured file as the input:
and assumes that the output:
is produced when the specified command is run.
These are all the supported keyword arguments:
input
the input file name. If it's not specified in configuration mode, all the variables in theconfiguration:
object (see above) are written to theoutput:
file.output
the output file name. In configuration mode, the permissions of the input file (if it is specified) are copied to the output file.configuration
as explained above, this is where you pass the configuration data object as returned byconfiguration_data()
command
as explained above, if specified, Meson does not create the file itself but rather runs the specified command, which allows you to do fully custom file generationinstall_dir
the subdirectory to install the generated file to (e.g.share/myproject
), if omitted the file is not installed.
custom_target
ctarget custom_target(*name*, ...)
Create a custom top level build target. The only positional argument is the name of this target and the keyword arguments are the following.
input
list of source filesoutput
list of output filescommand
command to run to create outputs from inputs. The command may be strings or the return offind_program()
orexecutable()
(note: always specify commands in array form['commandname', '-arg1', '-arg2']
rather than as a string'commandname -arg1 -arg2'
as the latter will not work)install
when true, this target is installed during the install stepinstall_dir
directory to install tobuild_always
iftrue
this target is always considered out of date and is rebuilt every time, useful for things such as build timestamps or revision control tagscapture
, there are some compilers that can't be told to write their output to a file but instead write it to standard output. When this argument is set to true, Meson capturesstdout
and writes it to the target file. Note that your command argument list may not contain@OUTPUT@
when capture mode is active.depends
specifies that this target depends on the specified target(s), even though it does not take any of them as a command line argument. This is meant for cases where you have a tool that e.g. does globbing internally. Usually you should just put the generated sources as inputs and Meson will set up all dependencies automatically.depend_files
files (string
,files()
, orconfigure_file()
) that this target depends on but are not listed in thecommand
kwarg. Useful for adding regen dependencies.depfile
is a dependency file that the command can write listing all the additional files this target depends on, for example a C compiler would list all the header files it included, and a change in any one of these files triggers a recompilationbuild_by_default
(added 0.38.0) causes, when set to true, to have this target be built by default, that is, when invoking plainninja
; the default value is false
The list of strings passed to the command
kwarg accept the following special string substitutions:
@INPUT@
the full path to the input passed toinput
. If more than one input is specified, all of them will be substituted as separate arguments only if the command uses'@INPUT@'
as a standalone-argument. For instance, this would not work:command : ['cp', './@INPUT@']
, but this would:command : ['cp', '@INPUT@']
.@OUTPUT@
the full path to the output passed tooutput
. If more than one outputs are specified, the behaviour is the same as@INPUT@
.@INPUT0@
@INPUT1@
...
the full path to the input with the specified array index ininput
@OUTPUT0@
@OUTPUT1@
...
the full path to the output with the specified array index inoutput
@OUTDIR@
the full path to the directory where the output(s) must be written@DEPFILE@
the full path to the dependency file passed todepfile
declare_dependency
dependency_object declare_dependency(...)
This function returns a dependency object that behaves like the return value of dependency
but is internal to the current build. The main use case for this is in subprojects. This allows a subproject to easily specify how it should be used. This makes it interchangeable with the same dependency that is provided externally by the system. This function has three keyword arguments.
include_directories
, the directories to add to header search pathlink_with
, libraries to link againstsources
, sources to add to targets (or generated header files that should be built before sources including them are built)dependencies
, other dependencies needed to use this dependencycompile_args
, compile arguments to uselink_args
, link arguments to useversion
, the version of this depency, such as1.2.3
dependency
dependency_object dependency(*dependency_name*, ...)
Finds an external dependency with the given name with pkg-config
if possible and with fallback detection logic otherwise. Dependency supports the following keyword arguments.
modules
specifies submodules to use for dependencies such as Qt5 or Boost.required
, when set to false, Meson will proceed with the build even if the dependency is not foundversion
, specifies the required version, a string containing a comparison operator followed by the version string, examples include>1.0.0
,<=2.3.5
or3.1.4
for exact matching. (Added 0.37.0) You can also specify multiple restrictions by passing a list to this kwarg, such as:['>=3.14.0', '<=4.1.0']
.native
if set totrue
, causes Meson to find the dependency on the build machine system rather than the host system (i.e. where the cross compiled binary will run on), usually only needed if you build a tool to be used during compilation.static
tells the dependency provider to try to get static libraries instead of dynamic ones (note that this is not supported by all dependency backends)fallback
specifies a subproject fallback to use in case the dependency is not found in the system. The value is an array['subproj_name', 'subproj_dep']
where the first value is the name of the subproject and the second is the variable name in that subproject that contains the value ofdeclare_dependency
.default_options
(added 0.37.0) an array of option values that override those set in the project'sdefault_options
invocation (likedefault_options
inproject()
, they only have effect when Meson is run for the first time, and command line arguments override any default options in build files)
The returned object also has methods that are documented in the object methods section below.
error
void error(message)
Print the argument string and halts the build process.
environment
environment_object environment()
Returns an empty environment variable object.
executable
exe executable(*exe_name*, *sources*, ...)
Creates a new executable. The first argument specifies its name and the remaining positional arguments define the source files to use. They can be of the following types:
- Strings relative to the current source directory
files()
objects defined in any preceding build file- The return value of configure-time generators such as
configure_file()
- The return value of build-time generators such as
custom_target()
orgenerator.process()
These source files can be sources, objects, libraries, or any other file. Meson will automatically categorize them based on the extension and use them accordingly. For instance, sources (.c
, .cpp
, .vala
, .rs
, etc) will be compiled, objects (.o
, .obj
) and libraries (.so
, .dll
, etc) will be linked, and all other files (headers, unknown extensions, etc) will be ignored. Meson will still create a build-time dependency on all source files, which is useful with generated files.
Executable supports the following keyword arguments. Note that just like the positional arguments above, these keyword arguments can also be passed to shared and static libraries.
link_with
, one or more shared or static libraries (built by this project) that this target should be linked with<languagename>_pch
precompiled header file to use for the given language<languagename>_args
compiler flags to use for the given language; eg:cpp_args
for C++link_args
flags to use during linking. You can use unix-style flags here for all platforms.link_depends
an extra file in the source tree that the link step depends on such as a symbol visibility map. The purpose is to automatically trigger a re-link (but not a re-compile) of the target when this file changes.include_directories
one or more objects created with theinclude_directories
functiondependencies
one or more objects created withdependency
orfind_library
(for external deps) ordeclare_dependency
(for deps built by the project)gui_app
when set to true flags this target as a GUI application on platforms where this makes a difference (e.g. Windows)extra_files
are not used for the build itself but are shown as source files in IDEs that group files by targets (such as Visual Studio)install
, when set to true, this executable should be installedinstall_rpath
a string to set the target's rpath to after install (but not before that)install_dir
override install directory for this file. The value is relative to theprefix
specified. F.ex, if you want to install plugins into a subdir, you'd use something like this:install_dir : get_option('libdir') + '/projectname-1.0'
.objects
list of prebuilt object files (usually for third party products you don't have source to) that should be linked in this target, never use this for object files that you build yourself.name_suffix
the string that will be used as the extension for the target by overriding the default. By default on Windows this isexe
and on other platforms it is omitted.build_by_default
causes, when set to true, to have this target be built by default, that is, when invoking plainninja
, the default value is true for all built target types, since 0.38.0
The list of sources
, objects
, and dependencies
is always flattened, which means you can freely nest and add lists while creating the final list. As a corollary, the best way to handle a 'disabled dependency' is by assigning an empty list []
to it and passing it like any other dependency to the dependencies:
kwarg.
The returned object also has methods that are documented in the object methods section below.
find_library
This function is deprecated and in the 0.31.0 release it was moved to the compiler object as obtained from meson.get_compiler(lang)
.
find_program
program find_program(program_name1, program_name2, ...)
program_name1
here is a string that can be an executable or script to be searched for in PATH
, or a script in the current source directory.
Meson will also autodetect scripts with a shebang line and run them with the executable/interpreter specified in it both on Windows (because the command invocator will reject the command otherwise) and UNIXes (if the script file does not have the executable bit set). Hence, you must not manually add the interpreter while using this script as part of a list of commands.
program_name2
and later positional arguments are used as fallback strings to search for. This is meant to be used for cases where the program may have many alternative names, such as foo
and foo.py
. The function will check for the arguments one by one and the first one that is found is returned. Meson versions earlier than 0.37.0 only accept one argument.
If none of the programs are found, Meson will abort. You can tell it not to by setting the keyword argument required
to false, and then use the .found()
method on the returned object to check whether it was found or not.
The returned object also has methods that are documented in the object methods section below.
files
file_array files(list_of_filenames)
This command takes the strings given to it in arguments and returns corresponding File objects that you can use as sources for build targets. The difference is that file objects remember the subdirectory they were defined in and can be used anywhere in the source tree. As an example suppose you have source file foo.cpp
in subdirectory bar1
and you would like to use it in a build target that is defined in bar2
. To make this happen you first create the object in bar1
like this:
foofile = files('foo.cpp')
Then you can use it in bar2
like this:
executable('myprog', 'myprog.cpp', foofile, ...)
Meson will then do the right thing.
generator
generator_object gen(*executable*, ...)
See also: custom_target
This function creates a generator object that can be used to run custom compilation commands. The only positional argument is the executable to use. It can either be a self-built executable or one returned by find_program. Keyword arguments are the following:
arguments
a list of template strings that will be the command line arguments passed to the executableoutput
a template string (or list of template strings) defining how an output file name is (or multiple output names are) generated from a single source file namedepfile
is a template string pointing to a dependency file that a generator can write listing all the additional files this target depends on, for example a C compiler would list all the header files it included, and a change in any one of these files triggers a recompilation
The returned object also has methods that are documented in the object methods section below.
The template strings passed to all the above kwargs accept the following special substitutions:
@PLAINNAME@
: the complete input file name, e.g:foo.c
becomesfoo.c
(unchanged)@BASENAME@
: the base of the input filename, e.g.:foo.c.y
becomesfoo.c
(extension is removed)
Each string passed to the outputs
kwarg must be constructed using one or both of these two substitutions.
In addition to the above substitutions, the arguments
kwarg also accepts the following:
@OUTPUT@
: the full path to the output file@INPUT@
: the full path to the input file@SOURCE_DIR@
: the full path to the root of the source tree@BUILD_DIR@
: the full path to the root of the build dir where the output will be placed
NOTE: Generators should only be used for outputs that will only be used as inputs for a build target or a custom target. When you use the processed output of a generator in multiple targets, the generator will be run multiple times to create outputs for each target. Each output will be created in a target-private directory @BUILD_DIR@
.
If you want to generate files for general purposes such as for generating headers to be used by several sources, or data that will be installed, and so on, use a custom_target
instead.
get_option
value get_option(option_name)
Obtains the value of the [project build option](Build options) specified in the positional argument.
get_variable
value get_variable(variable_name, fallback)
This function can be used to dynamically obtain a variable. res = get_variable(varname, fallback)
takes the value of varname
(which must be a string) and stores the variable of that name into res
. If the variable does not exist, the variable fallback
is stored to res
instead. If a fallback is not specified, then attempting to read a non-existing variable will cause a fatal error.
import
module_object import(module_name)
Imports the given extension module. Returns an opaque object that can be used to call the methods of the module. Here's an example for a hypothetical testmod
module.
tmod = import('testmod')
tmod.do_something()
include_directories
include_object include_directories(directory_names, ...)
Returns an opaque object which contains the directories given in positional arguments. The result can then be used as a keyword argument when building executables or libraries. Both the source directory and the corresponding build directory are added. Note that this function call itself does not add the directories into the search path, since there is no global search path. You can use the the returned object in any subdirectory you want, Meson will make the paths work automatically. This function has one keyword argument is_system
which, if set, flags the specified directories as system directories. This means that they will be used with the -isystem
compiler argument rather than -I
on compilers that support this flag (in practice everything except Visual Studio).
install_data
void install_data(list_of_files, ...)
Installs files from the source tree that are listed as positional arguments. The following keyword arguments are supported:
-
install_dir
the absolute or relative path to the installation directory. If this is a relative path, it is assumed to be relative to the prefix. -
install_mode
specify the file mode in symbolic format and optionally the owner/uid and group/gid for the installed files. For example:install_mode: 'rw-r--r--'
for just the file modeinstall_mode: ['rw-r--r--', 'nobody', 'nobody']
for the file mode and the user/groupinstall_mode: ['rw-r-----', 0, 0]
for the file mode and uid/gid
To leave any of these three as the default, specify false
.
install_headers
void install_headers(list_of_headers, ...)
Installs the specified header files from the source tree into the system header directory (usually /{prefix}/include
) during the install step. This directory can be overridden by specifying it with the install_dir
keyword argument. If you just want to install into a subdirectory of the system header directory, then use the subdir
argument. As an example if this has the value myproj
then the headers would be installed to /{prefix}/include/myproj
.
For example, this will install common.h
and kola.h
into /{prefix}/include
:
install_headers('common.h', 'proj/kola.h')
This will install common.h
and kola.h
into /{prefix}/include/myproj
:
install_headers('common.h', 'proj/kola.h', subdir : 'myproj')
This will install common.h
and kola.h
into /{prefix}/cust/myproj
:
install_headers('common.h', 'proj/kola.h', install_dir : 'cust', subdir : 'myproj')
install_man
void install_man(list_of_manpages, ...)
Installs the specified man files from the source tree into system's man directory during the install step. This directory can be overridden by specifying it with the install_dir
keyword argument.
install_subdir
void install_subdir(subdir_name)
Installs the entire given subdirectory and its contents from the source tree to the location specified by the keyword argument install_dir
. Note that due to implementation issues this command deletes the entire target dir before copying the files, so you should never use install_subdir
to install into two overlapping directories (such as foo
and foo/bar
) because if you do the behaviour is undefined.
is_variable
bool is_variable(varname)
Returns true if a variable of the given name exists and false otherwise.
jar
jar_object jar(name, list_of_sources, ...)
Build a jar from the specified Java source files. Keyword arguments are the same as executable's, with the addition of main_class
which specifies the main class to execute when running the jar with java -jar file.jar
.
join_paths
string join_paths(string1, string2, ...)
Joins the given strings into a file system path segment. For example join_paths('foo', 'bar')
results in foo/bar
. If any one of the individual segments is an absolute path, all segments before it are dropped. That means that join_paths('foo', '/bar')
returns /bar
.
Added 0.36.0
library
buildtarget library(library_name, list_of_sources, ...)
Builds a library that is either static or shared depending on the value of default_library
user option. You should use this instead of shared_library
or static_library
most of the time. This allows you to toggle your entire project (including subprojects) from shared to static with only one option.
The keyword arguments for this are the same as for executable
with the following addition:
name_prefix
the string that will be used as the suffix for the target by overriding the default (only used for libraries). By default this islib
on all platforms and compilers except with MSVC where it is omitted.
static_library
and shared_library
also accept this keyword argument.
message
void message(text)
This function prints its argument to stdout.
project
void project(project_name, list_of_languages, ...)
The first argument to this function must be a string defining the name of this project. It must be followed by one or more programming languages that the project uses. Supported values for languages are c
, cpp
(for C++
), objc
, objcpp
, fortran
, java
, cs
(for C#
) and vala
.
The project name can be any string you want, it's not used for anything except descriptive purposes. However since it is written to e.g. the dependency manifest is usually makes sense to have it be the same as the project tarball or pkg-config name. So for example you would probably want to use the name libfoobar instead of The Foobar Library.
Project supports the following keyword arguments.
-
version
, which is a free form string describing the version of this project. You can access the value in your Meson build files withmeson.project_version()
. -
subproject_dir
specifies the top level directory name that holds Meson subprojects. This is only meant as a compatibility option for existing code bases that house their embedded source code in a custom directory. All new projects should not set this but instead use the default value. It should be noted that this keyword argument is ignored inside subprojects. There can be only one subproject dir and it is set in the top level Meson file. -
meson_version
takes a string describing which Meson version the project requires. Usually something like>0.28.0
. Similarly you can specify the license(s) the code is under with thelicense
keyword argument. Usually this would be something likelicense : 'GPL2+'
, but if the code has multiple licenses you can specify them as an array like this:license : ['proprietary', 'GPL3']
. Note that the text is informal and is only written to the dependency manifest. Meson does not do any license validation, you are responsible for verifying that you abide by all licensing terms. -
default_options
takes an array of strings. The strings are in the formkey=value
and have the same format as options tomesonconf
. For example to set the default project type you would set this:default_options : ['buildtype=debugoptimized']
. Note that these settings are only used when running Meson for the first time. Global options such asbuildtype
can only be specified in the master project, settings in subprojects are ignored. Project specific options are used normally even in subprojects.
run_command
runresult run_command(command, list_of_args)
Runs the command specified in positional arguments. Returns an opaque object containing the result of the invocation. The script is run from an unspecified directory, and Meson will set three environment variables MESON_SOURCE_ROOT
, MESON_BUILD_ROOT
and MESON_SUBDIR
that specify the source directory, build directory and subdirectory the target was defined in, respectively.
run_target
buildtarget run_target(target_name, ...)
This function creates a new top-level target that runs a specified command with the specified arguments. Like all top-level targets, this integrates with the selected backend. For instance, with Ninja you can run it as ninja target_name
.
The script is run from an unspecified directory, and Meson will set three environment variables MESON_SOURCE_ROOT
, MESON_BUILD_ROOT
and MESON_SUBDIR
that specify the source directory, build directory and subdirectory the target was defined in, respectively.
command
is a list containing the command to run and the arguments to pass to it. Each list item may be a string or a target. For instance, passing the return value ofexecutable()
as the first item will run that executable, or passing a string as the first item will find that command inPATH
and run it.depends
is a list of targets that this target depends on but which are not listed in the command array (because, for example, the script does file globbing internally)
set_variable
void set_variable(variable_name, value)
Assigns a value to the given variable name. Calling set_variable('foo', bar)
is equivalent to foo = bar
.
shared_library
buildtarget shared_library(library_name, list_of_sources, ...)
Builds a shared library with the given sources. Positional and keyword arguments are the same as for library
with the following extra keyword arguments.
version
a string specifying the version of this shared library, such as1.1.0
. On Linux and OS X, this is used to set the shared library version in the filename, such aslibfoo.so.1.1.0
andlibfoo.1.1.0.dylib
. If this is not specified,soversion
is used instead (see below).soversion
a string specifying the soversion of this shared library, such as0
. On Linux and Windows this is used to set the soversion (or equivalent) in the filename. For example, ifsoversion
is4
, a Windows DLL will be calledfoo-4.dll
and one of the aliases of the Linux shared library would belibfoo.so.4
. If this is not specified, the first part ofversion
is used instead. For example, ifversion
is3.6.0
andsoversion
is not defined, it is set to3
.vs_module_defs
a string pointing to a file that contains Visual Studio symbol export definitions.
shared_module
buildtarget shared_module(module_name, list_of_sources, ...)
Builds a shared module with the given sources. Positional and keyword arguments are the same as for library
.
This is useful for building modules that will be dlopen()
ed and hence may contain undefined symbols that will be provided by the library that is loading it.
Added 0.37.0
static_library
buildtarget static_library(library_name, list_of_sources, ...)
Builds a static library with the given sources. Positional and keyword arguments are otherwise the same as for library
, but it has one argument the others don't have:
pic
, (Added 0.36.0) builds the library as positional independent code (so it can be linked into a shared library). This option has no effect on Windows and OS X since it doesn't make sense on Windows and PIC cannot be disabled on OS X.
subdir
void subdir(dir_name)
Enters the specified subdirectory and executes the meson.build
file in it. Once that is done, it returns and execution continues on the line following this subdir()
command. Variables defined in that meson.build
file are then available for use in later parts of the current build file and in all subsequent build files executed with subdir()
.
Note that this means that each meson.build
file in a source tree can and must only be executed once.
subproject
subproject_object subproject(subproject_name, ...)
Takes the project specified in the positional argument and brings that in the current build specification by returning a subproject object. Subprojects must always be placed inside the subprojects
directory at the top source directory. So for example a subproject called foo
must be located in ${MESON_SOURCE_ROOT}/subprojects/foo
. Supports the following keyword arguments:
version
keyword argument that works just like the one independency
. It specifies what version the subproject should be, as an example>=1.0.1
default_options
, (added 0.37.0) an array of default option values that override those set in the project'sdefault_options
invocation (likedefault_options
inproject
, they only have effect when Meson is run for the first time, and command line arguments override any default options in build files)
test
void test(name, executable, ...)
Defines an unit test. Takes two positional arguments, the first is the name of this test and the second is the executable to run. Keyword arguments are the following.
args
arguments to pass to the executableenv
environment variables to set, such as['NAME1=value1', 'NAME2=value2']
is_parallel
when false, specifies that no other test must be running at the same time as this testshould_fail
when true the test is considered passed if the executable returns a non-zero return value (i.e. reports an error)valgrind_args
if the test is run under Valgrind, pass these arguments to Valgrind (and not to the executable itself)timeout
the amount of seconds the test is allowed to run, a test that exceeds its time limit is always considered failed, defaults to 30 secondsworkdir
absolute path that will be used as the working directory for the test
Defined tests can be run in a backend-agnostic way by calling mesontest
inside the build dir, or by using backend-specific commands, such as ninja test
or msbuild RUN_TESTS.vcxproj
.
vcs_tag
ctarget vcs_tag(...)
This command detects revision control commit information at build time and places it in the specified output file. This file is guaranteed to be up to date on every build. Keywords are similar to custom_target
and all of them are mandatory.
input
file to modify (e.g.version.c.in
)output
file to write the results to (e.g.version.c
)fallback
version number to use when no revision control information is present, such as when building from a release tarball
Meson will read the contents of input
, replace the string @VCS_TAG@
with the detected revision number and write the result to output
. This method returns an opaque custom_target
object that you should put in your main program. If you desire more specific behavior than what this command provides, you should use custom_target
.
Object methods
Meson has several different object types that have methods users can call. This section describes them.
meson object
The meson
object allows you to introspect various properties of the system. This object is always mapped in the meson
variable. It has the following methods.
-
get_compiler(language)
returns an object describing a compiler, takes one positional argument which is the language to use. It also accepts one keyword argument,native
which when set to true makes Meson return the compiler for the build machine (the "native" compiler) and when false it returns the host compiler (the "cross" compiler). Ifnative
is omitted, Meson returns the "cross" compiler if we're currently cross-compiling and the "native" compiler if we're not. -
backend()
(added 0.37.0) returns a string representing the current backend:ninja
,vs2010
,vs2015
, orxcode
. -
is_cross_build()
returnstrue
if the current build is a cross build andfalse
otherwise -
is_unity()
returnstrue
when doing a unity build andfalse
otherwise -
is_subproject()
returnstrue
if the current project is being built as a subproject of some other project andfalse
otherwise -
has_exe_wrapper()
returns true when doing a cross build if there is a wrapper command that can be used to execute cross built binaries (for example when cross compiling from Linux to Windows, one can usewine
as the wrapper) -
add_install_script(script_name, arg1, arg2, ...)
causes the script given as an argument to be run during the install step, this script will have the environment variablesMESON_SOURCE_ROOT
,MESON_BUILD_ROOT
,MESON_INSTALL_PREFIX
, andMESON_INSTALL_DESTDIR_PREFIX
set. All additional arguments are passed as parameters.MESON_INSTALL_PREFIX
has the value of theprefix
option passed to Meson, andMESON_INSTALL_DESTDIR_PREFIX
has both the value ofDESTDIR
andprefix
joined together. This is useful because both are absolute paths, and many path-joining functions such asos.path.join
in Python special-case absolute paths. -
add_postconf_script(script_name, arg1, arg2, ...)
will run the executable given as an argument after all project files have been generated. This script will have the environment variablesMESON_SOURCE_ROOT
andMESON_BUILD_ROOT
set. -
current_source_dir()
returns a string to the current source directory. Note: you do not need to use this function when passing files from the current source directory to a function since that is the default. Also, you can use thefiles()
function to refer to files in the current or any other source directory instead of constructing paths manually withmeson.current_source_dir()
. -
current_build_dir()
returns a string to the current build directory -
source_root()
returns a string with the absolute path to the source root directory. Note: you should use thefiles()
function to refer to files in the root source directory instead of constructing paths manually withmeson.source_root()
. -
build_root()
returns a string with the absolute path to the build root directory -
project_version()
returns the version string specified inproject
function call -
project_name()
returns the project name specified in theproject
function call -
version()
return a string with the version of Meson -
get_cross_property(propname, fallback_value)
returns the given property from a cross file, the optional second argument is returned if not cross compiling or the given property is not found -
install_dependency_manifest(output_name)
installs a manifest file containing a list of all subprojects, their versions and license files to the file name given as the argument
build_machine object
Provides information about the build machine — the machine that is doing the actual compilation. See Cross-compilation. It has the following methods:
cpu_family()
returns the CPU family name. Guaranteed to returnx86
for 32-bit userland on x86 CPUs,x86_64
for 64-bit userland on x86 CPUs,arm
for 32-bit userland on all ARM CPUs, etc.cpu()
returns a more specific CPU name, such asi686
,amd64
, etc.system()
returns the operating system name, such aswindows
(all versions of Windows),linux
(all Linux distros),darwin
(all versions of OS X), etc.endian()
returnsbig
on big-endian systems andlittle
on little-endian systems.
If you think the returned values for any of these are incorrect for your system or CPU, please file a bug report.
host_machine object
Provides information about the host machine — the machine on which the compiled binary will run. See Cross-compilation.
It has the same methods as build_machine
, but note that it simply returns the values defined in the cross-info file.
target_machine object
Provides information about the target machine — the machine on which the compiled binary's output will run. Hence, this object should only be used while cross-compiling a compiler. See Cross-compilation.
It has the same methods as build_machine
, but note that it simply returns the values defined in the cross-info file. If target_machine
values are not defined in the cross-info file, host_machine
values are returned instead.
build target object
A build target is either an executable, shared or static library.
-
extract_objects()
returns an opaque value representing the generated object files of arguments, usually used to take single object files and link them to unit tests or to compile some source files with custom flags. To use the object file(s) in another build target, use theobjects:
keyword argument. -
extract_all_objects()
is same as above but returns all object files generated by this target -
private_dir_include()
returns a opaque value that works likeinclude_directories
but points to the private directory of this target, usually only needed if an another target needs to access some generated internal headers of this target -
full_path()
returns a full path pointing to the result target file
compiler object
This object is returned by meson.get_compiler(lang)
. It represents a compiler for a given language and allows you to query its properties. It has the following methods:
get_id()
returns a string identifying the compiler (e.g.'gcc'
)version()
returns the compiler's version number as a stringfind_library(lib_name, ...)
tries to find the library specified in the positional argument. The result object can be used just like the return value ofdependency
. If the keyword argumentrequired
is false, Meson will proceed even if the library is not found. By default the library is searched for in the system library directory (e.g. /usr/lib). This can be overridden with thedirs
keyword argument, which can be either a string or a list of strings.sizeof(typename, ...)
returns the size of the given type (e.g.'int'
) or -1 if the type is unknown, to add includes set them in theprefix
keyword argument, you can specify external dependencies to use withdependencies
keyword argumentalignment(typename)
returns the alignment of the type specified in the positional argument, you can specify external dependencies to use withdependencies
keyword argumentcompiles(code)
returns true if the code fragment given in the positional argument compiles, you can specify external dependencies to use withdependencies
keyword argument,code
can be either a string containing source code or afile
object pointing to the source codelinks(code)
returns true if the code fragment given in the positional argument compiles and links, you can specify external dependencies to use withdependencies
keyword argument,code
can be either a string containing source code or afile
object pointing to the source coderun(code)
attempts to compile and execute the given code fragment, returns a run result object, you can specify external dependencies to use withdependencies
keyword argument,code
can be either a string containing source code or afile
object pointing to the source codehas_header
returns true if the specified header can be included, you can specify external dependencies to use withdependencies
keyword argument and extra code to put above the header test with theprefix
keyword. In order to look for headers in a specific directory you can useargs : '-I/extra/include/dir
, but this should only be used in exceptional cases for includes that can't be detected via pkg-config and passed viadependencies
.has_type(typename)
returns true if the specified token is a type, you can specify external dependencies to use withdependencies
keyword argumenthas_function(funcname)
returns true if the given function is provided by the standard library or a library passed in with theargs
keyword, you can specify external dependencies to use withdependencies
keyword argumenthas_member(typename, membername)
takes two arguments, type name and member name and returns true if the type has the specified member, you can specify external dependencies to use withdependencies
keyword argumenthas_members(typename, membername1, membername2, ...)
takes at least two arguments, type name and one or more member names, returns true if the type has all the specified members, you can specify external dependencies to use withdependencies
keyword argumenthas_header_symbol(headername, symbolname)
allows one to detect whether a particular symbol (function, variable, #define, type definition, etc) is declared in the specified header, you can specify external dependencies to use withdependencies
keyword argumenthas_argument(argument_name)
returns true if the compiler accepts the specified command line argument, that is, can compile code without erroring out or printing a warning about an unknown flag, you can specify external dependencies to use withdependencies
keyword argumenthas_multi_arguments(arg1, arg2, arg3, ...)
is the same ashas_argument
but takes multiple arguments and uses them all in a single compiler invocation, available since 0.37.0first_supported_argument(list_of_strings)
, given a list of strings, returns the first argument that passes thehas_argument
test above or an empty array if none passsymbols_have_underscore_prefix()
returnstrue
if the C symbol mangling is one underscore (_
) prefixed to the symbol, available since 0.37.0compute_int(expr, ...')
computes the value of the given expression (as an example1 + 2
). When cross compiling this is evaluated with an iterative algorithm, you can specify keyword argumentslow
(defaults to -1024),high
(defaults to 1024) andguess
to specify max and min values for the search and the value to try first.
The following keyword arguments can be used:
-
name
the name to use for printing a message about the compiler check. Supported by the methodscompiles()
,links()
, andrun()
. If this kwarg is not passed to those methods, no message will be printed about the check. -
prefix
can be used to add #includes and other things that are required for the symbol to be declared. System definitions should be passed via compiler args (eg:_GNU_SOURCE
is often required for some symbols to be exposed on Linux, and it should be passed viaargs
keyword argument, see below). Supported by the methodssizeof
,has_type
,has_function
,has_member
,has_members
,has_header_symbol
. -
include_directories
specifies extra directories for header searches. (added 0.38.0) -
args
can be used to pass a list of compiler arguments that are required to find the header or symbol. For example, you might need to pass the include path-Isome/path/to/header
if a header is not in the default include path. In versions newer than 0.38.0 you should use theinclude_directories
keyword described above. You may also want to pass a library name-lfoo
forhas_function
to check for a function. Supported by all methods exceptget_id
,version
, andfind_library
.
Note that if you have a single prefix with all your dependencies, you might find it easier to append to the environment variables C_INCLUDE_PATH
with gcc/clang and INCLUDE
with msvc to expand the default include path, and LIBRARY_PATH
with gcc/clang and LIB
with msvc to expand the default library search path.
However, with GCC, these variables will be ignored when cross-compiling. In that case you need to use a specs file. See: http://www.mingw.org/wiki/SpecsFileHOWTO
run result object
This object encapsulates the result of trying to compile and run a sample piece of code with compiler.run()
. It has the following methods:
compiled()
if true, the compilation succeeded, if false it did not and the other methods return unspecified datareturncode()
the return code of executing the compiled binarystdout()
the standard out produced when the binary was runstderr()
the standard error produced when the binary was run
configuration data object
This object is returned by configuration_data
and encapsulates configuration values to be used for generating configuration files. A more in-depth description can be found in the the configuration wiki page It has three methods:
set(varname, value)
, sets a variable to a given valueset10(varname, boolean_value)
is the same as above but the value is eithertrue
orfalse
and will be written as 1 or 0, respectivelyset_quoted(varname, value)
is same asset
but quotes the value in double quotes ("
)has(varname)
, returnstrue
if the specified variable is setget(varname, default_value)
returns the value ofvarname
, if the value has not been set returnsdefault_value
if it is defined (added 0.38.0) and errors out if not
They all take the description
keyword that will be written in the result file. The replacement assumes a file with C syntax. If your generated file is source code in some other language, you probably don't want to add a description field because it most likely will cause a syntax error.
dependency object
This object is returned by dependency
and contains an external dependency with the following methods:
found()
which returns whether the dependency was foundtype_name()
which returns a string describing the type of the dependency, the most common values areinternal
for deps created withdeclare_dependencies
andpkgconfig
for system dependencies obtained with Pkg-config.version()
is the version number as a string, for example1.2.8
get_pkgconfig_variable(varname)
(Added 0.36.0) will get the pkg-config variable specified, or, if invoked on a non pkg-config dependency, error out
external program object
This object is returned by find_program
and contains an external (i.e. not built as part of this project) program and has the following methods:
found()
which returns whether the executable was foundpath()
which returns an array pointing to the executable (this is an array as opposed to a string because the program might be['python', 'foo.py']
, for example)
environment object
This object is returned by environment
and stores detailed information about how environment variables should be set during tests. It should be passed as the env
keyword argument to tests. It has the following methods.
set(varname, value)
sets environment variable in the first argument to the value in the second argument, e.g.env.set('FOO', 'BAR') sets envvar
FOOto value
BAR`append(varname, value)
appends the given value to the old value of the environment variable, e.g.env.append'('FOO', 'BAR', separator : ';')
producesBOB;BAR
ifFOO
had the valueBOB
and plainBAR
if the value was not defined. If the separator is not specified explicitly, the default path separator for the host operating system will be used, i.e. ';' for Windows and ':' for UNIX/POSIX systems.prepend(varname, value)
is the same asappend
except that it writes to the beginning of the variable
external library object
This object is returned by find_library
and contains an external (i.e. not built as part of this project) library. This object has only one method, found
, which returns whether the library was found.
generator object
This object is returned by generator
and contains a generator that is used to transform files from one type to another by an executable (e.g. idl
files into source code and headers).
process(list_of_files)
takes a list of files, causes them to be processed and returns an object containing the result which can then, for example, be passed into a build target definition. The keyword argumentextra_args
, if specified, will be used to replace an entry@EXTRA_ARGS@
in the argument list.
subproject object
This object is returned by subproject
and is an opaque object representing it.
get_variable(name)
fetches the specified variable from inside the subproject. This is useful to, for instance, get a declared dependency from the subproject.
string object
All strings have the following methods. Strings are immutable, all operations return their results as a new string.
strip()
removes whitespace at the beginning and end of the stringformat()
formats text, see the [[Syntax manual|Syntax#string-formatting]] for usage infoto_upper()
creates an upper case version of the stringto_lower()
creates a lower case version of the stringunderscorify()
creates a string where every non-alphabetical non-number character is replaced with_
split(split_character)
splits the string at the specified character (or whitespace if not set) and returns the parts in an arraystartswith(string)
returns true if string starts with the string specified as the argumentendswith(string)
returns true if string ends with the string specified as the argumentcontains(string)
returns true if string contains the string specified as the argumentto_int
returns the string converted to an integer (error if string is not a number)join(list_of_strings)
is the opposite of split, for example'.'.join(['a', 'b', 'c']
yields'a.b.c'
version_compare(comparison_string)
does semantic version comparison, ifx = '1.2.3'
thenx.version_compare('>1.0.0')
returnstrue
Number object
Numbers support these methods:
is_even()
returns true if the number is evenis_odd()
returns true if the number is odd
boolean object
A boolean object has two simple methods:
to_string()
returns the string'true'
if the boolean is true or'false'
otherwise. You can also pass it two strings as positional arguments to specify what to return for true/false. For instance,bool.to_string('yes', 'no')
will returnyes
if the boolean is true andno
if it is false.to_int()
as above, but returns either1
or0
array object
The following methods are defined for all arrays:
length()
, the size of the arraycontains(item)
, returnstrue
if the array contains the object given as argument,false
otherwiseget(index, fallback)
, returns the object at the given index, negative indices count from the back of the array, indexing out of bounds returns thefallback
value (added 0.38.0) or, if it is not specified, causes a fatal error
You can also iterate over arrays with the foreach
statement.
The results of the search are


Available Duties search duties
13
March
HMP Pentenville
13
March
HMP Pentenville
13
March
HMP Pentenville
13
March
HMP Pentenville
My next 3 Duties manage duties
13
March
HMP Pentenville
13
March
HMP Pentenville
13
March
HMP Pentenville


|
leaf-disk-gen
Leaf disk generation for DIRSIG scenes
|
Leaf disk generation for DIRSIG scenes.
This program generates a cloud of disjoint, disk-shaped “leaves” matching an underlying distribution of leaf angles, and outputs the resulting geometry either as a DIRSIG GList of true disk primitives or a Wavefront OBJ of triangulated disks. This is useful for abstract canopy simulations, which focus on overall light transport phenomena (not photorealism). The general program usage is
where desc
is a string describing the leaf angle distribution, [OPTIONS]
specifies global program options, and the sequence of box
or sphere
subcommands with [BOX-OPTIONS]
or [SPHERE-OPTIONS]
options specifies axis-aligned bounding boxes or spheres in which to generate the leaf primitives.
As mentioned, the desc
string describes the leaf angle distribution. There are currently four types of leaf angle distributions implemented.
"Uniform"
."Trigonometric TYPE"
where TYPE
is either Planophile
, Erectophile
, Plagiophile
, Extremophile
, or Spherical
. See the doxygen doc for mathematical expressions for each type."VerhoefBimodal A B"
where A
and B
are real numbers such that |A| + |B| <= 1
."NAME ALPHAX ALPHAY"
where NAME
is either TrowbridgeReitz
or Beckmann
, and ALPHAX
and ALPHAY
are positive floating point numbers corresponding to surface roughness in the X and Y directions. If these are equal, then the angle distribution is isotropic. Otherwise, the angle distribution is anisotropic.The global options [OPTIONS]
are as follows.
-s/--seed
to specify the seed for the random number generator. By default, this is 0
.-m/--matid
to specify the material ID number to assign to the leaf primitives. By default, this is 100
.-l/--lai
to specify Leaf Area Index (LAI) with respect to “ground” in the XY plane. LAI is defined as leaf area per unit of ground area. So, this indirectly controls the number of leaves generated. By default, this is 1
.-r/--radius
to specify the radius of leaf disks in meters. By default, this is 0.05
.-o/--output
to specify the output filename. This must end in either .glist
or .obj
, to designate the file as a DIRSIG GList or Wavefront OBJ respectively. By default, this is leaf.glist
.-ov/--output-ver-res
to specify the output vertex resolution. This is the number of vertices generated on the perimeter of each triangulated disk. This only affects Wavefront OBJ output, since DIRSIG GList output uses a true disk primitive. By default, this is 6
.-h/--help
to display program help, which includes brief descriptions of all program options.The box options [BOX-OPTIONS]
include only 2 options, --from
and --to
, which specify the coordinates of the corners of the box. Importantly, these options accept a string (surrounded by quotes) of an array (surrounded by square brackets, separated by commas) of three real numbers. For example, --from "[1, 2, 3]"
specifies the point X=1, Y=2, Z=3, for the “from” corner.
The sphere options [SPHERE-OPTIONS]
also include only 2 options, --center
and --radius
, which specify the center coordinate and radius of the sphere respectively.
As a more complete example,
generates a DIRSIG GList file verhoef-canopy.glist
containing disk primitive instances over -5<X<5
, -5<Y<5
, 0<Z<1
, with average LAI of 1.2, radius of 5 centimeters, and angles matching a Verhoef bimodal LIDF with A=-0.3
and B=0.2
.


TODO
The options are:
TODO stuff is available here:
Notice that TODO is still disposed to update. At least I would try and support its aplsimple.github.io version in a working state.
The links about and around TODO:


You are responsible for safeguarding the password that you use to access the Services and for any
activities or actions under your password. We encourage you to use "strong" passwords
(passwords that use a combination of upper and lower case letters, numbers and symbols)
with your account.


Where are my packages?¶
In this lesson we will learn to answer the following questions:
where are my installed packages located
what packages and package versions do I have
FYI: where is the standard library located?¶
We can use inspect.getfile(object)
to get the filename where a module, class or function is defined
import os, inspect
stdlib = os.path.dirname(inspect.getfile(os))
print(stdlib) # c:\users\aviad\appdata\local\programs\python\python37-32\lib
c:\users\aviad\appdata\local\programs\python\python37\lib
Package locations¶
pip
installs packages into a site-packages
directory in one of three locations
global packages - these packages all available for all users on the machine
per-user packages - these packages are available only to the current user
venv packages - virtual environment packages available to a particular environment/project
more on venv later
1. global site-packages¶
We you can use the site
module to figure out programatically where our global packages are being installed
import site, pprint
pprint.pprint(site.getsitepackages())
['c:\\users\\aviad\\appdata\\local\\programs\\python\\python37',
'c:\\users\\aviad\\appdata\\local\\programs\\python\\python37\\lib\\site-packages']
what packages (global or user) do I have?
!pip list -v
Package Version Location Installer
----------------------------- --------- ----------------------------------------------------------------------- ---------
alabaster 0.7.12 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
appdirs 1.4.4 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
argon2-cffi 20.1.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
arrow 0.17.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
astroid 2.2.5 c:\users\aviad\appdata\roaming\python\python37\site-packages pip
async-generator 1.10 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
atomicwrites 1.4.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
attrs 19.3.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
Babel 2.8.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
backcall 0.1.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
beautifulsoup4 4.9.3 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
binaryornot 0.4.4 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
bleach 3.1.3 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
bs4 0.0.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages
certifi 2020.6.20 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
cffi 1.14.2 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
chardet 3.0.4 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
click 7.1.2 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
colorama 0.4.4 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
cookiecutter 1.7.2 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
dataclasses 0.6 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
decorator 4.4.2 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
defusedxml 0.6.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
distlib 0.3.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
docutils 0.16 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
entrypoints 0.3 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
fastcore 1.0.11 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
fastscript 1.0.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
filelock 3.0.12 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
gitdb 4.0.5 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
GitPython 3.1.9 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
idna 2.10 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
imagesize 1.2.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
importlib-metadata 1.5.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
iniconfig 1.1.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
ipykernel 5.2.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
ipython 7.13.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
ipython-genutils 0.2.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
ipywidgets 7.5.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
isort 4.3.21 c:\users\aviad\appdata\roaming\python\python37\site-packages pip
jedi 0.16.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
Jinja2 2.11.2 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
jinja2-time 0.2.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
2. per-user site-package¶
Lets use the site
module again to see where our per-user site-packages folder is
# programmatically with python
site.getusersitepackages()
json5 0.9.5 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
jsonschema 3.2.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
jupyter 1.0.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
jupyter-book 0.8.3 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
jupyter-cache 0.4.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
jupyter-client 6.1.7 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
jupyter-console 6.1.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
jupyter-core 4.6.3 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
jupyter-sphinx 0.3.2 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
jupyterlab 2.2.8 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
jupyterlab-server 1.2.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
latexcodec 2.0.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
lazy-object-proxy 1.4.1 c:\users\aviad\appdata\roaming\python\python37\site-packages pip
markdown-it-py 0.5.5 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
MarkupSafe 1.1.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
mccabe 0.6.1 c:\users\aviad\appdata\roaming\python\python37\site-packages pip
mistune 0.8.4 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
myst-nb 0.10.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
myst-parser 0.12.10 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
nbclient 0.5.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
nbconvert 5.6.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
nbdev 1.0.18 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
nbdime 2.1.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
nbformat 5.0.4 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
nest-asyncio 1.4.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
notebook 6.1.3 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
numpy 1.19.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
oset 0.1.3 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
packaging 20.4 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
pandocfilters 1.4.2 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
parso 0.6.2 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
pickleshare 0.7.5 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
pip 20.2.3 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
pipenv 2020.8.13 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
pluggy 0.13.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
poyo 0.5.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
prometheus-client 0.7.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
prompt-toolkit 3.0.4 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
py 1.9.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
pybtex 0.23.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
pybtex-docutils 0.2.2 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
pycparser 2.20 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
pydata-sphinx-theme 0.4.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
Pygments 2.7.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
pylint 2.3.1 c:\users\aviad\appdata\roaming\python\python37\site-packages pip
pyparsing 2.4.7 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
pyrsistent 0.15.7 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages
pytest 6.1.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
pytest-sugar 0.9.4 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
python-dateutil 2.8.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
python-slugify 4.0.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
pytz 2020.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
pywin32 227 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
pywinpty 0.5.7 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
PyYAML 5.3.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
pyzmq 19.0.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
qtconsole 4.7.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
QtPy 1.9.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
requests 2.24.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
Send2Trash 1.5.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
setuptools 50.3.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
six 1.15.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
smmap 3.0.4 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
snowballstemmer 2.0.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
soupsieve 2.0.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
Sphinx 3.2.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
sphinx-book-theme 0.0.38 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
sphinx-comments 0.0.3 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
sphinx-copybutton 0.3.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
sphinx-panels 0.5.2 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
sphinx-thebe 0.0.7 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
sphinx-togglebutton 0.2.2 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
sphinxcontrib-applehelp 1.0.2 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
sphinxcontrib-bibtex 1.0.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
sphinxcontrib-devhelp 1.0.2 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
sphinxcontrib-htmlhelp 1.0.3 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
sphinxcontrib-jsmath 1.0.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
sphinxcontrib-qthelp 1.0.3 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
sphinxcontrib-serializinghtml 1.1.4 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
SQLAlchemy 1.3.20 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
style 1.1.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
termcolor 1.1.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
terminado 0.8.3 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
testpath 0.4.4 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
text-unidecode 1.3 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
toml 0.10.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
tornado 6.0.4 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
traitlets 4.3.3 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
typed-ast 1.4.0 c:\users\aviad\appdata\roaming\python\python37\site-packages pip
'C:\\Users\\aviad\\AppData\\Roaming\\Python\\Python37\\site-packages'
update 0.0.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
urllib3 1.25.10 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
virtualenv 20.0.35 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
virtualenv-clone 0.5.4 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
wcwidth 0.1.8 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
webencodings 0.5.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
wheel 0.35.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
widgetsnbextension 3.5.1 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
wrapt 1.11.2 c:\users\aviad\appdata\roaming\python\python37\site-packages
zipp 3.1.0 c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages pip
# from the shell / command-line
!python -m site --user-site
C:\Users\aviad\AppData\Roaming\Python\Python37\site-packages
what local packages do I have?
!pip list --user -v
Package Version Location Installer
----------------- ------- ------------------------------------------------------------ ---------
astroid 2.2.5 c:\users\aviad\appdata\roaming\python\python37\site-packages pip
isort 4.3.21 c:\users\aviad\appdata\roaming\python\python37\site-packages pip
lazy-object-proxy 1.4.1 c:\users\aviad\appdata\roaming\python\python37\site-packages pip
mccabe 0.6.1 c:\users\aviad\appdata\roaming\python\python37\site-packages pip
pylint 2.3.1 c:\users\aviad\appdata\roaming\python\python37\site-packages pip
typed-ast 1.4.0 c:\users\aviad\appdata\roaming\python\python37\site-packages pip
wrapt 1.11.2 c:\users\aviad\appdata\roaming\python\python37\site-packages
Development of Projects with Different Dependencies¶
Lets talk about three common issues that arise when using python extensively:
working with muliple projects, where each has their own depencies
having reproducible deterministic development / production environments
updating dependencies to fix security issues
we’re going to show one awesome tool called pipenv that is a solution to all three problems.
first, lets install pipenv:
!pip install pipenv
Requirement already satisfied: pipenv in c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages (2020.8.13)
Requirement already satisfied: pip>=18.0 in c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages (from pipenv) (20.2.3)
Requirement already satisfied: setuptools>=36.2.1 in c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages (from pipenv) (50.3.1)
Requirement already satisfied: virtualenv in c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages (from pipenv) (20.0.35)
Requirement already satisfied: virtualenv-clone>=0.2.5 in c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages (from pipenv) (0.5.4)
Requirement already satisfied: certifi in c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages (from pipenv) (2020.6.20)
Requirement already satisfied: six<2,>=1.9.0 in c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages (from virtualenv->pipenv) (1.15.0)
Requirement already satisfied: importlib-metadata<3,>=0.12; python_version < "3.8" in c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages (from virtualenv->pipenv) (1.5.0)
Requirement already satisfied: appdirs<2,>=1.4.3 in c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages (from virtualenv->pipenv) (1.4.4)
Requirement already satisfied: distlib<1,>=0.3.1 in c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages (from virtualenv->pipenv) (0.3.1)
Requirement already satisfied: filelock<4,>=3.0.0 in c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages (from virtualenv->pipenv) (3.0.12)
Requirement already satisfied: zipp>=0.5 in c:\users\aviad\appdata\local\programs\python\python37\lib\site-packages (from importlib-metadata<3,>=0.12; python_version < "3.8"->virtualenv->pipenv) (3.1.0)
1.working with muliple projects, where each has their own dependencies¶
Lets examine a common issue that arises when you’re working on multiple projects. Imagine that ProjectA needs django==2.1
, but ProjectB needs django==1.9
.
By default, Python tries to store all your third-party packages in the system-wide location. This means that every time you want to switch between ProjectA and ProjectB, you have to make sure the right version of django is installed. This makes switching between projects painful because you have to uninstall and reinstall packages to meet the requirements for each project.
The standard solution is to use a virtual environment that has its own Python executable and third-party package storage. That way, ProjectA and ProjectB are adequately separated. Now you can easily switch between projects since they’re not sharing the same package storage location. PackageA can have whatever version of django it needs in its own environment, and PackageB can have what it needs totally separate.
a common, slightly older way to fix this issue is using virtualenv + virtualenvwrapper.
see this stackoverflow answer for a nice comparison of tools
Pipenv has virtual environment management built in so that you have a single tool for your package management.
Lets see how to use this in practice, by creating two seperate django projects
django is a well-loved web framework
run the following in your shell
>> mkdir ProjectA
>> cd ProjectA
>> pipenv install django==2.1
>> pipenv shell
(ProjectA) >> django-admin startproject ProjectA .
(ProjectA) >> exit
pipenv did a few things for us:
when we ran
pipenv install django==2.1
it detected that we’re in a new projectit created a new, empty virtual environment for us, where there are no pre-installed python packages
it created a new file called
Pipfile
for us
thePipfile
contains a specification of all of our direct dependenciesit added a dependency on django version 2.1 to the Pipfile
it downloaded and installed django 2.1 to our virtual environment
running
pipenv shell
allowed us to enter our virtual environment where our PATH variable is set to include only packages in the virtual environment. theit created a
Pipfile.lock
file which lists all the sub-dependencies django relies on (as it happens, django does not rely on additional packages)
this is how the file structure looks now
ProjectA/
├── ProjectA/
├── manage.py
├── Pipfile
└── Pipfile.lock
Notes:
we should commit to source control the
Pipfile
andPipfile.lock
files that describe’s the project dependencies.notice that running
pip freeze
, django was not installed in our global/user directory, but rather just in our virtual environmentwe could now easily create as many projects as we want with different dependencies or versions of dependencies without any collisions
2. having reproducible deterministic development / production environments¶
Now lets imagine we’re using the pandas
package in our project.
right now we’re just interested in having an up-to-date version without caring too much about the version.
lets add it to our environment
>> pipenv install pandas
as it happens, the pandas package relies on a few sub-dependencies like numpy
>> pipenv graph
Django==2.1
- pytz [required: Any, installed: 2019.1]
pandas==0.24.2
- numpy [required: >=1.12.0, installed: 1.16.4]
- python-dateutil [required: >=2.5.0, installed: 2.8.0]
- six [required: >=1.5, installed: 1.12.0]
- pytz [required: >=2011k, installed: 2019.1]
but what happens if one of the sub-dependencies later publish a new version with breaking changes, and another developer on my team tries to install the project’s dependencies and ends up with a build that doesn’t work?
It worked on my computer, I swear
What we need is a deterministic build - that is, to be able to have exactly the same environment and code for the project on our teammates’ computer, as well as in production. this is exactly the information held in our Pipfile.lock
file. it has a list of all the sub-dependencies that were installed, and their exact version numbers and hashes.
We can install the exact packages and versions listed in our Pipfile.lock file using the pipenv sync
command.
>> pipenv sync
Installing dependencies from Pipfile.lock (aac533)…
================================ 6/6 - 00:00:03
All dependencies are now up-to-date!
when deploying code to developer machines or to production environment pipenv sync
creates a deterministic build by ensuring everyone has exactly the same packages in the same versions
3. updating dependencies to fix security issues¶
so for our ProjectA we pinned the version of django to be 2.1 django==2.1
.
we have a deterministic build and our code seems to work well.
but what if there was an important bugfix or a security error in one of our dependencies, or worse, in one of our sub-dependencies?
lets check for security issues by running pipenv check
:
(ProjectA) >> pipenv check
Checking PEP 508 requirements…
Passed!
Checking installed package safety…
36883: django <2.1.6,>=2.1.0 resolved (2.1 installed)!
Django 2.1.x before 2.1.6 allows Uncontrolled Memory Consumption via a malicious attacker-supplied value to the django.utils.numberformat.format() function.
36522: django <2.1.2,>=2.1 resolved (2.1 installed)!
An issue was discovered in Django 2.1 before 2.1.2, in which unprivileged users can read the password hashes of arbitrary accounts. The read-only password widget used by the Django Admin to display an obfuscated password hash was bypassed if a user has only the "view" permission (new in Django 2.1), resulting in display of the entire password hash to those users. This may result in a vulnerability for sites with legacy user accounts using insecure hashes.
36517: django <2.1.2,>=2.1.0 resolved (2.1 installed)!
django before 2.1.2 fixes a security bug in 2.1.x.
If an admin user has the change permission to the user model, only part of the
password hash is displayed in the change form. Admin users with the view (but
not change) permission to the user model were displayed the entire hash.
Oh My!
seems a lot has already been been fixed since 2.1.0, the version we pinned to …
lets upgrade to the latest bugfix version of 2.1.x
>> pipenv install django<2.2
this should do it. lets take a look at our list of packages now
Django==2.1.10
- pytz [required: Any, installed: 2019.1]
pandas==0.24.2
- numpy [required: >=1.12.0, installed: 1.16.4]
- python-dateutil [required: >=2.5.0, installed: 2.8.0]
- six [required: >=1.5, installed: 1.12.0]
- pytz [required: >=2011k, installed: 2019.1]
now we have the latest django 2.1.10, without switching to a major new django 2.2.x lets see if our security problems are fixed:
>> pipenv check
Checking PEP 508 requirements…
Passed!
Checking installed package safety…
All good!
what happens if in the future additional security issues are fixed for the 2.1 branch?
running pipenv install
would install all packages mentioned in my Pipfile
, including upgrading them if newer versions that satisfy my constraints are available.
Tying it all together¶
so what have we seen?
pipenv manages a virtual environment for us where each project can have exactly the dependencies it needs and nothing more
we used
pipenv install django
andpipenv install pandas
to add packages to our virtual environmentwe used two PEP 508 version specifiers to pin down a version of django.
django==2.1
which pinned us to 2.1.0 anddjango<2.2
which allowed us to receive bug/security fixes upto 2.1.10we used
pipenv sync
to install a deterministic build using the specific pinned down versions specified in ourPipfile.lock
filewe used
pipenv install
to upgrade to the latest versions in ourPipfile
that satisfy our constraintswe used
pipenv check
to test for security issueswe used
pipenv shell
to enter into our virtual environment and use installed tools for that environment like django-admin
generally, pipenv …
Enables truly deterministic builds, while easily specifying only what you want.
Generates and checks file hashes for locked dependencies.
Automatically generates a Pipfile, if one doesn’t exist.
Automatically creates a virtualenv in a standard location.
Automatically adds/removes packages to a Pipfile when they are un/installed.
Further reading¶
https://docs.pipenv.org/en/latest/
https://realpython.com/pipenv-guide/
https://www.python.org/dev/peps/pep-0508/
Additional features¶
development-only packages¶
Let’s say you also have some unit tests for this awesome application, and you want to use pytest for running them. You don’t need pytest
in production so you can specify that this dependency is only for development with the --dev
argument:
$ pipenv install pytest --dev
Providing the --dev
argument will put the dependency in a special [dev-packages] location in the Pipfile. These development packages only get installed if you specify the --dev
argument with pipenv install.
The different sections separate dependencies needed only for development from ones needed for the base code to actually work. Typically, this would be accomplished with additional requirements files like dev-requirements.txt or test-requirements.txt. Now, everything is consolidated in a single Pipfile under different sections.


Fever and rash in a child can have many causes. The majority of them are viral, and in an otherwise healthy child they are not usually life-threatening. However, if your child suddenly develops a fever along with the type of rash mentioned above, especially if he/she is acting abnormally in any way, emergency evaluation is needed. Even a one-hour delay could affect your child’s chances of survival. Get him or her to an emergency room immediately.
One cause is a bacterium called meningococcus (most common in children under two years old or between fifteen and twenty-four years of age), which produces a serious contagious infection in the blood that progresses rapidly. It often responds well to early treatment with intravenous antibiotics. Current recommendations are for adolescents to be immunized against meningococcosis between the ages of eleven and eighteen. However, the vaccines do not protect against all of the strains of this bacterium, and it may only offer protection for five to eight years.
A fever, flu-like symptoms, and a quickly developing rash can also be caused by other medical conditions, including inflammation of the blood vessels after a viral infection. Also, a similar-looking rash that usually develops more slowly and without a fever or other symptoms could be due to a reduced number of platelets in the blood. The child is at risk of severe bleeding, for instance a brain hemorrhage. (See Appendix D, for tips on using a thermometer.)


--- title: Visualising US Voting Records with shinydashboard author: Gokhan Ciflikli date: '2017-12-27' slug: adavis categories: - R - Visualization tags: - R - statebins - plotly - ggExtra - shinydashboard description: 'Visual exploration of nearly 70 years of US voting records' ---
My second ever post on this blog was on introducing adamap, a Shiny app that maps Americans for Democratic Action voting scores (the so-called Liberal Quotient) between 1947-2015. It was built with highcharter
, and hence it was nicely interactive but quite slow. I wanted to switch to another package since, and when I eventually ran into statebins
, I knew what had to be done.
I was certain that statebins
would definitely add some oomph to the design, but because it’s so easy to implement, I had some spare time to do other things. As it is often the case, one thing led to the other, and I came to the conclusion that the revamped app should feature one plot from every major graphics package. Of course, a strict implementation of that statement would be quite difficult, so I downgraded the challenge to just four plots using a different package each time. I ended with statebins
for state-level mapping, plotly
for plotting changes from the previous year, ggExtra
for head-to-head state comparisons, and *drum roll* base R for graphing a single politican’s voting record over time. It turned out to be fun.
As I have already explained the data setup in my previous post linked above, I will skip to the visualisations. I won’t be including the actual code for the Shiny app, which includes reactive elements throughout. You can fork the code underlying the live dashboard on GitHub and/or run a local copy via the runGitHub
code provided there. Also, none of the codes are evaluated here (as I translate them from the app), so they will not work if you just plug them in. Hence, there are primarily for motivation rather than replication.
There are many good things about statebins
. First, you get identical-sized states so you don’t get biased by the variation in their size. It’s cleaner by definition, not featuring idiosyncratic shapes that are found in nature. Also, it plays really nice with viridis
, which is important (maybe). In addition, you can define light and dark labels for the state abbreviations, ensuring they will not blend into the fill colour. statebins
can be called as a function (as I did), or applied later to a ggplot
object. The only thing that did not work for me was the ggplot2_scale_function
argument; R kept saying no such function is defined (I’m using the dev version from GitHub) so I ended up passing the fill colours separately. It gives a warning about overriding the existing fill, but works otherwise. If you download the dataset and want to visualise how House Democrats in 1962 voted, something along the lines of:
#Not evaluated
library(ggplot2)
library(viridis)
library(statebins)
library(hrbrthemes)
theme_set(theme_ipsum_rc())
#assumes data = data, year as "Year", state names in "State", voting scores in "ADA" etc.
us <- statebins(data[data$Year == 1962 & data$Chamber == 1 & data$Party == "Democrat", ],
state_col = "State", value_col = "ADA",
round = TRUE, font_size = 7, state_border_col = "#232d33",
dark_label = "white",
light_label = "black",
state_border_size = .8, radius = grid::unit(10, "pt")) +
labs(title = "") +
theme(axis.text.x = element_blank(), axis.text.y = element_blank(), axis.ticks = element_blank(),
panel.grid.minor = element_blank(), panel.grid.major = element_blank(),
panel.background = element_rect(fill = "white", linetype = "blank"),
legend.position = c(.075, .85),
legend.text = element_text(colour = "#232d33", size = 14),
legend.title = element_text(colour = "#232d33", size = 18),
legend.key.height = grid::unit(.01, "snpc"),
legend.key.width = grid::unit(.05, "snpc"),
plot.margin = margin(-1, 0, 0, 0, "cm"))
us + scale_fill_viridis(direction = -1,
breaks = c(seq(-25, 100, 25)),
labels = c("No Rep", paste(seq(0, 100, 25), "% ")),
guide = guide_legend(title = "", title.position = "top",
keywidth = 2, keyheight = 2, ncol = 1))
should do the trick. Now, I am totally cheating because the image is from the live app and the above code is not evaluated. However, it should give you an idea, mainly most of the clutter is about the layout rather than the content. Can we get someone with clout to ping Hadley and Bob regarding any updates on issue#4 from 2015 so we get magically created bins (and other shapes, for the ambitious) not just for US states but for everything?1 Cheers.
One trick regarding the data; the original data only have scores for selected representatives (naturally). Meaning, in any given year, there will be several states (approx. 8-10 per party) with no Democrat or Republican reps. As these are row-wise missing instead of NA
, if you plot them as they are, those states will not show in the plot. If only there was a tidyverse function that would solve common data problems like this…
library(tidyverse)
#Add rows for missing states in party-year
#Use built-in states data
states <- data.frame(state.name, stringsAsFactors = FALSE)
states$state.no <- 1:50
dataset <- merge(dataset, states, by.x = "State", by.y = "state.name")
#I couldn't get this to work with strings so matched them by state.no
dataset <- dataset %>% tidyr::complete(state.no = full_seq(state.no, period = 1), Year, Chamber, Party,
fill = list(ADA = -25, aADA = -25)) #Arbitrary low score instead of NA
dataset$State <- ifelse(is.na(dataset$State), states[dataset$state.no, 1], dataset$State)
Moving on to challenge number #2, I wanted to keep to the same filter (Year > Chamber > Party), but with the amount of change from last year plotted instead. I haven’t used plotly
much before so I learned on-the-go, but it has robust documentation if you are considering delving into it.
The main reason for going with plotly was its build-in interactivity. I wanted the users to just hover over points and see a block of text describing the shift from the previous year.  One curious thing was the alpha functionality, which is governed with toRGB("colour", "alpha")
, but called opacity in plotly
. In the app, the default is showing Senate Republicans in 1990 (i.e. difference from 1989):
library(plotly)
#Store common args
ax <- list(
showline = FALSE,
showticklabels = TRUE,
showgrid = FALSE)
#Mandatory data should be subsetted before comment
plot_ly(data, x = ~Score, key = data[, 1],
y = ~Change, mode = "markers", type = "scatter", hoverinfo = "text",
hoverlabel = list(font = list(family = "Roboto Condensed", size = 14)),
#Add custom hover text
text = ~paste(data$Chamber, data$Party, "from",
State, "voted\n", paste0(abs(round(Change, 2)), "% more"), Label, "in", data$Year),
color = ~Change, colors = viridis(direction = -1, n = 12),
marker = list(size = 30, opacity = .7)) %>%
layout(dragmode = "select", showlegend = FALSE,
xaxis = c(ax, list(title = "Selected Year Voting Score", zeroline = FALSE)),
yaxis = c(ax, list(title = "Change from Last Year", zeroline = TRUE, zerolinecolor = toRGB("black", .05))),
font = list(family = "Roboto Condensed")) %>%
config(displayModeBar = FALSE)
I admit today’s code chunks are a bit like this, so if you have any questions, just fire away.
This whole project is basically me putting together various plotting techniques that I like but had yet to implement. The histograms-on-the-side style of the ggExtra
package is definitely one of those. You can also have density plots (and box-plots) instead, which sounds like it could be more informative but IMHO it’s harder to read, especially without a baseline underneath. YMMV. The main function is ggMarginal
, and you have to call it on an existing ggplot
object - pretty straightforward.
With this graph, I wanted to compare the historical record of two states (still operating under the Congress > Party subset). One could argue this could have been more informative if it included both parties; say how Democrats and Republican from NY voted in 2002. Perhaps, however that might introduce inconsistencies with the design, especially with the (almost) global slider options. The default plots House Democrats from California and Texas from 1948 to 2015, which shows a clear separation:
library(ggExtra)
#data should be a two-state subset
p <- ggplot(data, aes(Year, Score, color = Score)) +
geom_point(aes(shape = State), size = 3, alpha = .7) +
scale_color_viridis(direction = -1) +
theme(legend.position = "none", axis.title.x = element_text(size = 20),
axis.title.y = element_text(size = 20), axis.text = element_text(size = 18))
ggMarginal(p, type = "histogram", margins = "y", yparams = list(
bins = 50, fill = viridis(n = 49, alpha = .8, direction = -1)))
When I saw this brilliant post on Tufte, the plot I wanted to replicate the most was the very first one. The one that was done in base R. Some might even argue I added a representative look-up tab to the app just for trying this out. Hmm. Like plotly
, I was out of practice with base R graphics, so I mimicked the original code as much as I could. One thing I wanted to convey with this graph is the consistency of a single politician over their tenure. I didn’t want to show minima and maxima, but just their mean score with some sort of confidence measure. I also learned that you can pass Greek letters with expression()
, which is handy. Say, you want to plot the complete voting history of Nancy Pelosi:
#Store descriptives
v1 <- mean(data$Score)
v2 <- sd(data$Score)
v3 <- min(data$Year)
v4 <- max(data$Year)
v5 <- summary(data$Year)
#Base plot, data should be a representative subset
plot(data$Score ~ data$Year, xlab = "", ylab = "", axes = FALSE, family = "Roboto Condensed",
pch = 16, type = "b", lwd = 2)
#Upper sd
abline(h = v1 + v2, lty = 2, col = alpha("black", .2))
#Mean
abline(h = v1, lty = 2)
#Lower sd
abline(h = v1 - v2, lty = 2, col = alpha("black", .2))
#Right axis
axis(1, at = c(v3, v4, (v3 + v4) / 2),
labels = c(v3, v4, round((v3 + v4) / 2, 0)),
tick = FALSE, family = "Roboto Condensed")
#Bottom axis
axis(2, at = c(v1, v1 + v2, v1 - v2), labels = round(c(v1, v1 + v2, v1 - v2), 0),
las = 2, family = "Roboto Condensed", tick = FALSE, lty = 0)
#Left axis
axis(4, at = c(v1, v1 + v2, v1 - v2), lwd = 0, las = 2,
labels = c(expression(mu), expression(sigma), expression(sigma)),
col = alpha("black", .2), family = "Roboto Condensed", ps = 20)
I might be relying on viridis
a lot, although I also utilise the RColorBrewer
package as well (Map of Westeros, anyone?). To be honest, I more or less only like the default palette, the namesake or option = "D"
, but others might fancy some good old diversity. To this end, I added a drop-down menu for switching viridis
palettes, and a button for changing the direction (i.e. whether the palette should start from the lightest or the darkest colour). Both of these options are global, so you can switch any time at any tab. Except for the base R plot; that looks much better in black, Tufte style.