Deployment Guide¶
Hardware Considerations¶
Swift is designed to run on commodity hardware. At Rackspace, our storage servers are currently running fairly generic 4U servers with 24 2T SATA drives and 8 cores of processing power. RAID on the storage drives is not required and not recommended. Swift’s disk usage pattern is the worst case possible for RAID, and performance degrades very quickly using RAID 5 or 6.
Deployment Options¶
The Swift services run completely autonomously, which provides for a lot of flexibility when architecting the hardware deployment for Swift. The 4 main services are:
- Proxy Services
- Object Services
- Container Services
- Account Services
The Proxy Services are more CPU and network I/O intensive. If you are using 10g networking to the proxy, or are terminating SSL traffic at the proxy, greater CPU power will be required.
The Object, Container, and Account Services (Storage Services) are more disk and network I/O intensive.
The easiest deployment is to install all services on each server. There is nothing wrong with doing this, as it scales each service out horizontally.
At Rackspace, we put the Proxy Services on their own servers and all of the Storage Services on the same server. This allows us to send 10g networking to the proxy and 1g to the storage servers, and keep load balancing to the proxies more manageable. Storage Services scale out horizontally as storage servers are added, and we can scale overall API throughput by adding more Proxies.
If you need more throughput to either Account or Container Services, they may each be deployed to their own servers. For example you might use faster (but more expensive) SAS or even SSD drives to get faster disk I/O to the databases.
A high-availability (HA) deployment of Swift requires that multiple proxy servers are deployed and requests are load-balanced between them. Each proxy server instance is stateless and able to respond to requests for the entire cluster.
Load balancing and network design is left as an exercise to the reader, but this is a very important part of the cluster, so time should be spent designing the network for a Swift cluster.
Web Front End Options¶
Swift comes with an integral web front end. However, it can also be deployed as a request processor of an Apache2 using mod_wsgi as described in Apache Deployment Guide.
Preparing the Ring¶
The first step is to determine the number of partitions that will be in the ring. We recommend that there be a minimum of 100 partitions per drive to insure even distribution across the drives. A good starting point might be to figure out the maximum number of drives the cluster will contain, and then multiply by 100, and then round up to the nearest power of two.
For example, imagine we are building a cluster that will have no more than 5,000 drives. That would mean that we would have a total number of 500,000 partitions, which is pretty close to 2^19, rounded up.
It is also a good idea to keep the number of partitions small (relatively). The more partitions there are, the more work that has to be done by the replicators and other backend jobs and the more memory the rings consume in process. The goal is to find a good balance between small rings and maximum cluster size.
The next step is to determine the number of replicas to store of the data. Currently it is recommended to use 3 (as this is the only value that has been tested). The higher the number, the more storage that is used but the less likely you are to lose data.
It is also important to determine how many zones the cluster should have. It is recommended to start with a minimum of 5 zones. You can start with fewer, but our testing has shown that having at least five zones is optimal when failures occur. We also recommend trying to configure the zones at as high a level as possible to create as much isolation as possible. Some example things to take into consideration can include physical location, power availability, and network connectivity. For example, in a small cluster you might decide to split the zones up by cabinet, with each cabinet having its own power and network connectivity. The zone concept is very abstract, so feel free to use it in whatever way best isolates your data from failure. Each zone exists in a region.
A region is also an abstract concept that may be used to distinguish between geographically separated areas as well as can be used within same datacenter. Regions and zones are referenced by a positive integer.
You can now start building the ring with:
swift-ring-builder <builder_file> create <part_power> <replicas> <min_part_hours>
This will start the ring build process creating the <builder_file> with 2^<part_power> partitions. <min_part_hours> is the time in hours before a specific partition can be moved in succession (24 is a good value for this).
Devices can be added to the ring with:
swift-ring-builder <builder_file> add r<region>z<zone>-<ip>:<port>/<device_name>_<meta> <weight>
This will add a device to the ring where <builder_file> is the name of the builder file that was created previously, <region> is the number of the region the zone is in, <zone> is the number of the zone this device is in, <ip> is the ip address of the server the device is in, <port> is the port number that the server is running on, <device_name> is the name of the device on the server (for example: sdb1), <meta> is a string of metadata for the device (optional), and <weight> is a float weight that determines how many partitions are put on the device relative to the rest of the devices in the cluster (a good starting point is 100.0 x TB on the drive).Add each device that will be initially in the cluster.
Once all of the devices are added to the ring, run:
swift-ring-builder <builder_file> rebalance
This will distribute the partitions across the drives in the ring. It is important whenever making changes to the ring to make all the changes required before running rebalance. This will ensure that the ring stays as balanced as possible, and as few partitions are moved as possible.
The above process should be done to make a ring for each storage service (Account, Container and Object). The builder files will be needed in future changes to the ring, so it is very important that these be kept and backed up. The resulting .tar.gz ring file should be pushed to all of the servers in the cluster. For more information about building rings, running swift-ring-builder with no options will display help text with available commands and options. More information on how the ring works internally can be found in the Ring Overview.
Running object-servers Per Disk¶
The lack of true asynchronous file I/O on Linux leaves the object-server workers vulnerable to misbehaving disks. Because any object-server worker can service a request for any disk, and a slow I/O request blocks the eventlet hub, a single slow disk can impair an entire storage node. This also prevents object servers from fully utilizing all their disks during heavy load.
Another way to get full I/O isolation is to give each disk on a storage node a different port in the storage policy rings. Then set the servers_per_port option in the object-server config. NOTE: while the purpose of this config setting is to run one or more object-server worker processes per disk, the implementation just runs object-servers per unique port of local devices in the rings. The deployer must combine this option with appropriately-configured rings to benefit from this feature.
Here’s an example (abbreviated) old-style ring (2 node cluster with 2 disks each):
Devices: id region zone ip address port replication ip replication port name
0 1 1 <IP_ADDRESS> 6200 <IP_ADDRESS> 6200 d1
1 1 1 <IP_ADDRESS> 6200 <IP_ADDRESS> 6200 d2
2 1 2 <IP_ADDRESS> 6200 <IP_ADDRESS> 6200 d3
3 1 2 <IP_ADDRESS> 6200 <IP_ADDRESS> 6200 d4
And here’s the same ring set up for servers_per_port:
Devices: id region zone ip address port replication ip replication port name
0 1 1 <IP_ADDRESS> 6200 <IP_ADDRESS> 6200 d1
1 1 1 <IP_ADDRESS> 6201 <IP_ADDRESS> 6201 d2
2 1 2 <IP_ADDRESS> 6200 <IP_ADDRESS> 6200 d3
3 1 2 <IP_ADDRESS> 6201 <IP_ADDRESS> 6201 d4
When migrating from normal to servers_per_port, perform these steps in order:
- Upgrade Swift code to a version capable of doing servers_per_port.
- Enable servers_per_port with a > 0 value
- Restart swift-object-server processes with a SIGHUP. At this point, you will have the servers_per_port number of swift-object-server processes serving all requests for all disks on each node. This preserves availability, but you should perform the next step as quickly as possible.
- Push out new rings that actually have different ports per disk on each server. One of the ports in the new ring should be the same as the port used in the old ring (“6200” in the example above). This will cover existing proxy-server processes who haven’t loaded the new ring yet. They can still talk to any storage node regardless of whether or not that storage node has loaded the ring and started object-server processes on the new ports.
If you do not run a separate object-server for replication, then this setting must be available to the object-replicator and object-reconstructor (i.e. appear in the [DEFAULT] config section).
General Service Configuration¶
Most Swift services fall into two categories. Swift’s wsgi servers and background daemons.
For more information specific to the configuration of Swift’s wsgi servers with paste deploy see General Server Configuration.
Configuration for servers and daemons can be expressed together in the same file for each type of server, or separately. If a required section for the service trying to start is missing there will be an error. The sections not used by the service are ignored.
Consider the example of an object storage node. By convention, configuration
for the object-server, object-updater, object-replicator, object-auditor, and
object-reconstructor exist in a single file /etc/swift/object-server.conf
:
[DEFAULT]
reclaim_age = 604800
[pipeline:main]
pipeline = object-server
[app:object-server]
use = egg:swift#object
[object-replicator]
[object-updater]
[object-auditor]
Swift services expect a configuration path as the first argument:
$ swift-object-auditor
Usage: swift-object-auditor CONFIG [options]
Error: missing config path argument
If you omit the object-auditor section this file could not be used as the
configuration path when starting the swift-object-auditor
daemon:
$ swift-object-auditor /etc/swift/object-server.conf
Unable to find object-auditor config section in /etc/swift/object-server.conf
If the configuration path is a directory instead of a file all of the files in the directory with the file extension ”.conf” will be combined to generate the configuration object which is delivered to the Swift service. This is referred to generally as “directory based configuration”.
Directory based configuration leverages ConfigParser’s native multi-file support. Files ending in ”.conf” in the given directory are parsed in lexicographical order. Filenames starting with ‘.’ are ignored. A mixture of file and directory configuration paths is not supported - if the configuration path is a file only that file will be parsed.
The Swift service management tool swift-init
has adopted the convention of
looking for /etc/swift/{type}-server.conf.d/
if the file
/etc/swift/{type}-server.conf
file does not exist.
When using directory based configuration, if the same option under the same section appears more than once in different files, the last value parsed is said to override previous occurrences. You can ensure proper override precedence by prefixing the files in the configuration directory with numerical values.:
/etc/swift/
default.base
object-server.conf.d/
000_default.conf -> ../default.base
001_default-override.conf
010_server.conf
020_replicator.conf
030_updater.conf
040_auditor.conf
You can inspect the resulting combined configuration object using the
swift-config
command line tool
General Server Configuration¶
Swift uses paste.deploy (http://pythonpaste.org/deploy/) to manage server configurations.
Default configuration options are set in the [DEFAULT] section, and any
options specified there can be overridden in any of the other sections BUT
ONLY BY USING THE SYNTAX set option_name = value
. This is the unfortunate
way paste.deploy works and I’ll try to explain it in full.
First, here’s an example paste.deploy configuration file:
[DEFAULT]
name1 = globalvalue
name2 = globalvalue
name3 = globalvalue
set name4 = globalvalue
[pipeline:main]
pipeline = myapp
[app:myapp]
use = egg:mypkg#myapp
name2 = localvalue
set name3 = localvalue
set name5 = localvalue
name6 = localvalue
The resulting configuration that myapp receives is:
global {'__file__': '/etc/mypkg/wsgi.conf', 'here': '/etc/mypkg',
'name1': 'globalvalue',
'name2': 'globalvalue',
'name3': 'localvalue',
'name4': 'globalvalue',
'name5': 'localvalue',
'set name4': 'globalvalue'}
local {'name6': 'localvalue'}
So, name1 got the global value which is fine since it’s only in the DEFAULT section anyway.
name2 got the global value from DEFAULT even though it appears to be overridden in the app:myapp subsection. This is just the unfortunate way paste.deploy works (at least at the time of this writing.)
name3 got the local value from the app:myapp subsection because it is using
the special paste.deploy syntax of set option_name = value
. So, if you want
a default value for most app/filters but want to override it in one
subsection, this is how you do it.
name4 got the global value from DEFAULT since it’s only in that section
anyway. But, since we used the set
syntax in the DEFAULT section even
though we shouldn’t, notice we also got a set name4
variable. Weird, but
probably not harmful.
name5 got the local value from the app:myapp subsection since it’s only
there anyway, but notice that it is in the global configuration and not the
local configuration. This is because we used the set
syntax to set the
value. Again, weird, but not harmful since Swift just treats the two sets of
configuration values as one set anyway.
name6 got the local value from app:myapp subsection since it’s only there,
and since we didn’t use the set
syntax, it’s only in the local
configuration and not the global one. Though, as indicated above, there is no
special distinction with Swift.
That’s quite an explanation for something that should be so much simpler, but it might be important to know how paste.deploy interprets configuration files. The main rule to remember when working with Swift configuration files is:
Note
Use the set option_name = value
syntax in subsections if the option is
also set in the [DEFAULT]
section. Don’t get in the habit of always
using the set
syntax or you’ll probably mess up your non-paste.deploy
configuration files.
Common configuration¶
An example of common configuration file can be found at etc/swift.conf-sample
The following configuration options are available:
| Option | Default | Description |
| max_header_size | 8192 | max_header_size is the max number of bytes in the utf8 encoding of each header. Using 8192 as default because eventlet use 8192 as max size of header line. This value may need to be increased when using identity v3 API tokens including more than 7 catalog entries. See also include_service_catalog in proxy-server.conf-sample (documented in overview_auth.rst). |
| extra_header_count | 0 | By default the maximum number of allowed headers depends on the number of max allowed metadata settings plus a default value of 32 for regular http headers. If for some reason this is not enough (custom middleware for example) it can be increased with the extra_header_count constraint. |
Object Server Configuration¶
An Example Object Server configuration can be found at etc/object-server.conf-sample in the source code repository.
The following configuration options are available:
[DEFAULT]
| Option | Default | Description |
| swift_dir | /etc/swift | Swift configuration directory |
| devices | /srv/node | Parent directory of where devices are mounted |
| mount_check | true | Whether or not check if the devices are mounted to prevent accidentally writing to the root device |
| bind_ip | <IP_ADDRESS> | IP Address for server to bind to |
| bind_port | 6200 | Port for server to bind to |
| bind_timeout | 30 | Seconds to attempt bind before giving up |
| backlog | 4096 | Maximum number of allowed pending connections |
| workers | auto | Override the number of pre-forked workers that will accept connections. If set it should be an integer, zero means no fork. If unset, it will try to default to the number of effective cpu cores and fallback to one. Increasing the number of workers helps slow filesystem operations in one request from negatively impacting other requests, but only the servers_per_port option provides complete I/O isolation with no measurable overhead. |
| servers_per_port | 0 | If each disk in each storage policy ring has unique port numbers for its “ip” value, you can use this setting to have each object-server worker only service requests for the single disk matching the port in the ring. The value of this setting determines how many worker processes run for each port (disk) in the ring. If you have 24 disks per server, and this setting is 4, then each storage node will have 1 + (24 * 4) = 97 total object-server processes running.  The object-replicator and object-reconstructor need to see this setting too, so it must be in the [DEFAULT] section. See Running object-servers Per Disk. |
| max_clients | 1024 | Maximum number of clients one worker can process simultaneously (it will actually accept(2) N + 1). Setting this to one (1) will only handle one request at a time, without accepting another request concurrently. |
| disable_fallocate | false | Disable “fast fail” fallocate checks if the underlying filesystem does not support it. |
| log_name | swift | Label used when logging |
| log_facility | LOG_LOCAL0 | Syslog log facility |
| log_level | INFO | Logging level |
| log_address | /dev/log | Logging directory |
| log_max_line_length | 0 | Caps the length of log lines to the value given; no limit if set to 0, the default. |
| log_custom_handlers | None | Comma-separated list of functions to call to setup custom log handlers. |
| log_udp_host | Override log_address | |
| log_udp_port | 514 | UDP log port |
| log_statsd_host | None | Enables StatsD logging; IPv4/IPv6 address or a hostname. If a hostname resolves to an IPv4 and IPv6 address, the IPv4 address will be used. |
| log_statsd_port | 8125 | |
| log_statsd_default_sample_rate | 1.0 | |
| log_statsd_sample_rate_factor | 1.0 | |
| log_statsd_metric_prefix | ||
| eventlet_debug | false | If true, turn on debug logging for eventlet |
| fallocate_reserve | 1% | You can set fallocate_reserve to the number of bytes or percentage of disk space you’d like fallocate to reserve, whether there is space for the given file size or not. Percentage will be used if the value ends with a ‘%’. This is useful for systems that behave badly when they completely run out of space; you can make the services pretend they’re out of space early. |
| conn_timeout | 0.5 | Time to wait while attempting to connect to another backend node. |
| node_timeout | 3 | Time to wait while sending each chunk of data to another backend node. |
| client_timeout | 60 | Time to wait while receiving each chunk of data from a client or another backend node |
| network_chunk_size | 65536 | Size of chunks to read/write over the network |
| disk_chunk_size | 65536 | Size of chunks to read/write to disk |
| container_update_timeout | 1 | Time to wait while sending a container update on object update. |
| reclaim_age | 604800 | Time elapsed in seconds before the tombstone file representing a deleted object can be reclaimed. This is the maximum window for your consistency engine. If a node that was disconnected from the cluster because of a fault is reintroduced into the cluster after this window without having its data purged it will result in dark data. This setting should be consistent across all object services. |
| nice_priority | None | Scheduling priority of server processes. Niceness values range from -20 (most favorable to the process) to 19 (least favorable to the process). The default does not modify priority. |
| ionice_class | None | I/O scheduling class of server processes. I/O niceness class values are IOPRIO_CLASS_RT (realtime), IOPRIO_CLASS_BE (best-effort), and IOPRIO_CLASS_IDLE (idle). The default does not modify class and priority. Linux supports io scheduling priorities and classes since 2.6.13 with the CFQ io scheduler. Work only with ionice_priority. |
| ionice_priority | None | I/O scheduling priority of server processes. I/O niceness priority is a number which goes from 0 to 7. The higher the value, the lower the I/O priority of the process. Work only with ionice_class. Ignored if IOPRIO_CLASS_IDLE is set. |
[object-server]
| Option | Default | Description |
| use | paste.deploy entry point for the object server. For most cases, this should be egg:swift#object. | |
| set log_name | object-server | Label used when logging |
| set log_facility | LOG_LOCAL0 | Syslog log facility |
| set log_level | INFO | Logging level |
| set log_requests | True | Whether or not to log each request |
| set log_address | /dev/log | Logging directory |
| user | swift | User to run as |
| max_upload_time | 86400 | Maximum time allowed to upload an object |
| slow | 0 | If > 0, Minimum time in seconds for a PUT or DELETE request to complete. This is only useful to simulate slow devices during testing and development. |
| mb_per_sync | 512 | On PUT requests, sync file every n MB |
| keep_cache_size | 5242880 | Largest object size to keep in buffer cache |
| keep_cache_private | false | Allow non-public objects to stay in kernel’s buffer cache |
| allowed_headers | Content-Disposition, Content-Encoding, X-Delete-At, X-Object-Manifest, X-Static-Large-Object | Comma separated list of headers that can be set in metadata on an object. This list is in addition to X-Object-Meta-* headers and cannot include Content-Type, etag, Content-Length, or deleted |
| auto_create_account_prefix | . | Prefix used when automatically creating accounts. |
| replication_server | Configure parameter for creating specific server. To handle all verbs, including replication verbs, do not specify “replication_server” (this is the default). To only handle replication, set to a True value (e.g. “True” or “1”). To handle only non-replication verbs, set to “False”. Unless you have a separate replication network, you should not specify any value for “replication_server”. | |
| replication_concurrency | 4 | Set to restrict the number of concurrent incoming SSYNC requests; set to 0 for unlimited |
| replication_one_per_device | True | Restricts incoming SSYNC requests to one per device, replication_currency above allowing. This can help control I/O to each device, but you may wish to set this to False to allow multiple SSYNC requests (up to the above replication_concurrency setting) per device. |
| replication_lock_timeout | 15 | Number of seconds to wait for an existing replication device lock before giving up. |
| replication_failure_threshold | 100 | The number of subrequest failures before the replication_failure_ratio is checked |
| replication_failure_ratio | 1.0 | If the value of failures / successes of SSYNC subrequests exceeds this ratio, the overall SSYNC request will be aborted |
| splice | no | Use splice() for zero-copy object GETs. This requires Linux kernel version 3.0 or greater. If you set “splice = yes” but the kernel does not support it, error messages will appear in the object server logs at startup, but your object servers should continue to function. |
| nice_priority | None | Scheduling priority of server processes. Niceness values range from -20 (most favorable to the process) to 19 (least favorable to the process). The default does not modify priority. |
| ionice_class | None | I/O scheduling class of server processes. I/O niceness class values are IOPRIO_CLASS_RT (realtime), IOPRIO_CLASS_BE (best-effort), and IOPRIO_CLASS_IDLE (idle). The default does not modify class and priority. Linux supports io scheduling priorities and classes since 2.6.13 with the CFQ io scheduler. Work only with ionice_priority. |
| ionice_priority | None | I/O scheduling priority of server processes. I/O niceness priority is a number which goes from 0 to 7. The higher the value, the lower the I/O priority of the process. Work only with ionice_class. Ignored if IOPRIO_CLASS_IDLE is set. |
[object-replicator]
| Option | Default | Description |
| log_name | object-replicator | Label used when logging |
| log_facility | LOG_LOCAL0 | Syslog log facility |
| log_level | INFO | Logging level |
| log_address | /dev/log | Logging directory |
| daemonize | yes | Whether or not to run replication as a daemon |
| interval | 30 | Time in seconds to wait between replication passes |
| concurrency | 1 | Number of replication workers to spawn |
| sync_method | rsync | The sync method to use; default is rsync but you can use ssync to try the EXPERIMENTAL all-swift-code-no-rsync-callouts method. Once ssync is verified as or better than, rsync, we plan to deprecate rsync so we can move on with more features for replication. |
| rsync_timeout | 900 | Max duration of a partition rsync |
| rsync_bwlimit | 0 | Bandwidth limit for rsync in kB/s. 0 means unlimited. |
| rsync_io_timeout | 30 | Timeout value sent to rsync –timeout and –contimeout options |
| rsync_compress | no | Allow rsync to compress data which is transmitted to destination node during sync. However, this is applicable only when destination node is in a different region than the local one. NOTE: Objects that are already compressed (for example: .tar.gz, .mp3) might slow down the syncing process. |
| stats_interval | 300 | Interval in seconds between logging replication statistics |
| handoffs_first | false | If set to True, partitions that are not supposed to be on the node will be replicated first. The default setting should not be changed, except for extreme situations. |
| handoff_delete | auto | By default handoff partitions will be removed when it has successfully replicated to all the canonical nodes. If set to an integer n, it will remove the partition if it is successfully replicated to n nodes. The default setting should not be changed, except for extreme situations. |
| node_timeout | DEFAULT or 10 | Request timeout to external services. This uses what’s set here, or what’s set in the DEFAULT section, or 10 (though other sections use 3 as the final default). |
| http_timeout | 60 | Max duration of an http request. This is for REPLICATE finalization calls and so should be longer than node_timeout. |
| lockup_timeout | 1800 | Attempts to kill all workers if nothing replicates for lockup_timeout seconds |
| rsync_module | {replication_ip}::object | Format of the rsync module where the replicator will send data. The configuration value can include some variables that will be extracted from the ring. Variables must follow the format {NAME} where NAME is one of: ip, port, replication_ip, replication_port, region, zone, device, meta. See etc/rsyncd.conf-sample for some examples. |
| rsync_error_log_line_length | 0 | Limits how long rsync error log lines are |
| ring_check_interval | 15 | Interval for checking new ring file |
| recon_cache_path | /var/cache/swift | Path to recon cache |
| nice_priority | None | Scheduling priority of server processes. Niceness values range from -20 (most favorable to the process) to 19 (least favorable to the process). The default does not modify priority. |
| ionice_class | None | I/O scheduling class of server processes. I/O niceness class values are IOPRIO_CLASS_RT (realtime), IOPRIO_CLASS_BE (best-effort), and IOPRIO_CLASS_IDLE (idle). The default does not modify class and priority. Linux supports io scheduling priorities and classes since 2.6.13 with the CFQ io scheduler. Work only with ionice_priority. |
| ionice_priority | None | I/O scheduling priority of server processes. I/O niceness priority is a number which goes from 0 to 7. The higher the value, the lower the I/O priority of the process. Work only with ionice_class. Ignored if IOPRIO_CLASS_IDLE is set. |
[object-updater]
| Option | Default | Description |
| log_name | object-updater | Label used when logging |
| log_facility | LOG_LOCAL0 | Syslog log facility |
| log_level | INFO | Logging level |
| log_address | /dev/log | Logging directory |
| interval | 300 | Minimum time for a pass to take |
| concurrency | 1 | Number of updater workers to spawn |
| node_timeout | DEFAULT or 10 | Request timeout to external services. This uses what’s set here, or what’s set in the DEFAULT section, or 10 (though other sections use 3 as the final default). |
| slowdown | 0.01 | Time in seconds to wait between objects |
| recon_cache_path | /var/cache/swift | Path to recon cache |
| nice_priority | None | Scheduling priority of server processes. Niceness values range from -20 (most favorable to the process) to 19 (least favorable to the process). The default does not modify priority. |
| ionice_class | None | I/O scheduling class of server processes. I/O niceness class values are IOPRIO_CLASS_RT (realtime), IOPRIO_CLASS_BE (best-effort), and IOPRIO_CLASS_IDLE (idle). The default does not modify class and priority. Linux supports io scheduling priorities and classes since 2.6.13 with the CFQ io scheduler. Work only with ionice_priority. |
| ionice_priority | None | I/O scheduling priority of server processes. I/O niceness priority is a number which goes from 0 to 7. The higher the value, the lower the I/O priority of the process. Work only with ionice_class. Ignored if IOPRIO_CLASS_IDLE is set. |
[object-auditor]
| Option | Default | Description |
| log_name | object-auditor | Label used when logging |
| log_facility | LOG_LOCAL0 | Syslog log facility |
| log_level | INFO | Logging level |
| log_address | /dev/log | Logging directory |
| log_time | 3600 | Frequency of status logs in seconds. |
| interval | 30 | Time in seconds to wait between auditor passes |
| disk_chunk_size | 65536 | Size of chunks read during auditing |
| files_per_second | 20 | Maximum files audited per second per auditor process. Should be tuned according to individual system specs. 0 is unlimited. |
| bytes_per_second | 10000000 | Maximum bytes audited per second per auditor process. Should be tuned according to individual system specs. 0 is unlimited. |
| concurrency | 1 | The number of parallel processes to use for checksum auditing. |
| zero_byte_files_per_second | 50 | |
| object_size_stats | ||
| recon_cache_path | /var/cache/swift | Path to recon cache |
| rsync_tempfile_timeout | auto | Time elapsed in seconds before rsync tempfiles will be unlinked. Config value of “auto” try to use object-replicator’s rsync_timeout + 900 or fallback to 86400 (1 day). |
| nice_priority | None | Scheduling priority of server processes. Niceness values range from -20 (most favorable to the process) to 19 (least favorable to the process). The default does not modify priority. |
| ionice_class | None | I/O scheduling class of server processes. I/O niceness class values are IOPRIO_CLASS_RT (realtime), IOPRIO_CLASS_BE (best-effort), and IOPRIO_CLASS_IDLE (idle). The default does not modify class and priority. Linux supports io scheduling priorities and classes since 2.6.13 with the CFQ io scheduler. Work only with ionice_priority. |
| ionice_priority | None | I/O scheduling priority of server processes. I/O niceness priority is a number which goes from 0 to 7. The higher the value, the lower the I/O priority of the process. Work only with ionice_class. Ignored if IOPRIO_CLASS_IDLE is set. |
Container Server Configuration¶
An example Container Server configuration can be found at etc/container-server.conf-sample in the source code repository.
The following configuration options are available:
[DEFAULT]
| Option | Default | Description |
| swift_dir | /etc/swift | Swift configuration directory |
| devices | /srv/node | Parent directory of where devices are mounted |
| mount_check | true | Whether or not check if the devices are mounted to prevent accidentally writing to the root device |
| bind_ip | <IP_ADDRESS> | IP Address for server to bind to |
| bind_port | 6201 | Port for server to bind to |
| bind_timeout | 30 | Seconds to attempt bind before giving up |
| backlog | 4096 | Maximum number of allowed pending connections |
| workers | auto | Override the number of pre-forked workers that will accept connections. If set it should be an integer, zero means no fork. If unset, it will try to default to the number of effective cpu cores and fallback to one. Increasing the number of workers may reduce the possibility of slow file system operations in one request from negatively impacting other requests. See General Service Tuning. |
| max_clients | 1024 | Maximum number of clients one worker can process simultaneously (it will actually accept(2) N + 1). Setting this to one (1) will only handle one request at a time, without accepting another request concurrently. |
| user | swift | User to run as |
| disable_fallocate | false | Disable “fast fail” fallocate checks if the underlying filesystem does not support it. |
| log_name | swift | Label used when logging |
| log_facility | LOG_LOCAL0 | Syslog log facility |
| log_level | INFO | Logging level |
| log_address | /dev/log | Logging directory |
| log_max_line_length | 0 | Caps the length of log lines to the value given; no limit if set to 0, the default. |
| log_custom_handlers | None | Comma-separated list of functions to call to setup custom log handlers. |
| log_udp_host | Override log_address | |
| log_udp_port | 514 | UDP log port |
| log_statsd_host | None | Enables StatsD logging; IPv4/IPv6 address or a hostname. If a hostname resolves to an IPv4 and IPv6 address, the IPv4 address will be used. |
| log_statsd_port | 8125 | |
| log_statsd_default_sample_rate | 1.0 | |
| log_statsd_sample_rate_factor | 1.0 | |
| log_statsd_metric_prefix | ||
| eventlet_debug | false | If true, turn on debug logging for eventlet |
| fallocate_reserve | 1% | You can set fallocate_reserve to the number of bytes or percentage of disk space you’d like fallocate to reserve, whether there is space for the given file size or not. Percentage will be used if the value ends with a ‘%’. This is useful for systems that behave badly when they completely run out of space; you can make the services pretend they’re out of space early. |
| db_preallocation | off | If you don’t mind the extra disk space usage in overhead, you can turn this on to preallocate disk space with SQLite databases to decrease fragmentation. |
| nice_priority | None | Scheduling priority of server processes. Niceness values range from -20 (most favorable to the process) to 19 (least favorable to the process). The default does not modify priority. |
| ionice_class | None | I/O scheduling class of server processes. I/O niceness class values are IOPRIO_CLASS_RT (realtime), IOPRIO_CLASS_BE (best-effort), and IOPRIO_CLASS_IDLE (idle). The default does not modify class and priority. Linux supports io scheduling priorities and classes since 2.6.13 with the CFQ io scheduler. Work only with ionice_priority. |
| ionice_priority | None | I/O scheduling priority of server processes. I/O niceness priority is a number which goes from 0 to 7. The higher the value, the lower the I/O priority of the process. Work only with ionice_class. Ignored if IOPRIO_CLASS_IDLE is set. |
[container-server]
| Option | Default | Description |
| use | paste.deploy entry point for the container server. For most cases, this should be egg:swift#container. | |
| set log_name | container-server | Label used when logging |
| set log_facility | LOG_LOCAL0 | Syslog log facility |
| set log_level | INFO | Logging level |
| set log_requests | True | Whether or not to log each request |
| set log_address | /dev/log | Logging directory |
| node_timeout | 3 | Request timeout to external services |
| conn_timeout | 0.5 | Connection timeout to external services |
| allow_versions | false | Enable/Disable object versioning feature |
| auto_create_account_prefix | . | Prefix used when automatically |
| replication_server | Configure parameter for creating specific server. To handle all verbs, including replication verbs, do not specify “replication_server” (this is the default). To only handle replication, set to a True value (e.g. “True” or “1”). To handle only non-replication verbs, set to “False”. Unless you have a separate replication network, you should not specify any value for “replication_server”. | |
| nice_priority | None | Scheduling priority of server processes. Niceness values range from -20 (most favorable to the process) to 19 (least favorable to the process). The default does not modify priority. |
| ionice_class | None | I/O scheduling class of server processes. I/O niceness class values are IOPRIO_CLASS_RT (realtime), IOPRIO_CLASS_BE (best-effort), and IOPRIO_CLASS_IDLE (idle). The default does not modify class and priority. Linux supports io scheduling priorities and classes since 2.6.13 with the CFQ io scheduler. Work only with ionice_priority. |
| ionice_priority | None | I/O scheduling priority of server processes. I/O niceness priority is a number which goes from 0 to 7. The higher the value, the lower the I/O priority of the process. Work only with ionice_class. Ignored if IOPRIO_CLASS_IDLE is set. |
[container-replicator]
| Option | Default | Description |
| log_name | container-replicator | Label used when logging |
| log_facility | LOG_LOCAL0 | Syslog log facility |
| log_level | INFO | Logging level |
| log_address | /dev/log | Logging directory |
| per_diff | 1000 | Maximum number of database rows that will be sync’d in a single HTTP replication request. Databases with less than or equal to this number of differing rows will always be sync’d using an HTTP replication request rather than using rsync. |
| max_diffs | 100 | Maximum number of HTTP replication requests attempted on each replication pass for any one container. This caps how long the replicator will spend trying to sync a given database per pass so the other databases don’t get starved. |
| concurrency | 8 | Number of replication workers to spawn |
| interval | 30 | Time in seconds to wait between replication passes |
| node_timeout | 10 | Request timeout to external services |
| conn_timeout | 0.5 | Connection timeout to external services |
| reclaim_age | 604800 | Time elapsed in seconds before a container can be reclaimed |
| rsync_module | {replication_ip}::container | Format of the rsync module where the replicator will send data. The configuration value can include some variables that will be extracted from the ring. Variables must follow the format {NAME} where NAME is one of: ip, port, replication_ip, replication_port, region, zone, device, meta. See etc/rsyncd.conf-sample for some examples. |
| rsync_compress | no | Allow rsync to compress data which is transmitted to destination node during sync. However, this is applicable only when destination node is in a different region than the local one. NOTE: Objects that are already compressed (for example: .tar.gz, mp3) might slow down the syncing process. |
| recon_cache_path | /var/cache/swift | Path to recon cache |
| nice_priority | None | Scheduling priority of server processes. Niceness values range from -20 (most favorable to the process) to 19 (least favorable to the process). The default does not modify priority. |
| ionice_class | None | I/O scheduling class of server processes. I/O niceness class values are IOPRIO_CLASS_RT (realtime), IOPRIO_CLASS_BE (best-effort), and IOPRIO_CLASS_IDLE (idle). The default does not modify class and priority. Linux supports io scheduling priorities and classes since 2.6.13 with the CFQ io scheduler. Work only with ionice_priority. |
| ionice_priority | None | I/O scheduling priority of server processes. I/O niceness priority is a number which goes from 0 to 7. The higher the value, the lower the I/O priority of the process. Work only with ionice_class. Ignored if IOPRIO_CLASS_IDLE is set. |
[container-updater]
| Option | Default | Description |
| log_name | container-updater | Label used when logging |
| log_facility | LOG_LOCAL0 | Syslog log facility |
| log_level | INFO | Logging level |
| log_address | /dev/log | Logging directory |
| interval | 300 | Minimum time for a pass to take |
| concurrency | 4 | Number of updater workers to spawn |
| node_timeout | 3 | Request timeout to external services |
| conn_timeout | 0.5 | Connection timeout to external services |
| slowdown | 0.01 | Time in seconds to wait between containers |
| account_suppression_time | 60 | Seconds to suppress updating an account that has generated an error (timeout, not yet found, etc.) |
| recon_cache_path | /var/cache/swift | Path to recon cache |
| nice_priority | None | Scheduling priority of server processes. Niceness values range from -20 (most favorable to the process) to 19 (least favorable to the process). The default does not modify priority. |
| ionice_class | None | I/O scheduling class of server processes. I/O niceness class values are IOPRIO_CLASS_RT (realtime), IOPRIO_CLASS_BE (best-effort), and IOPRIO_CLASS_IDLE (idle). The default does not modify class and priority. Linux supports io scheduling priorities and classes since 2.6.13 with the CFQ io scheduler. Work only with ionice_priority. |
| ionice_priority | None | I/O scheduling priority of server processes. I/O niceness priority is a number which goes from 0 to 7. The higher the value, the lower the I/O priority of the process. Work only with ionice_class. Ignored if IOPRIO_CLASS_IDLE is set. |
[container-auditor]
| Option | Default | Description |
| log_name | container-auditor | Label used when logging |
| log_facility | LOG_LOCAL0 | Syslog log facility |
| log_level | INFO | Logging level |
| log_address | /dev/log | Logging directory |
| interval | 1800 | Minimum time for a pass to take |
| containers_per_second | 200 | Maximum containers audited per second. Should be tuned according to individual system specs. 0 is unlimited. |
| recon_cache_path | /var/cache/swift | Path to recon cache |
| nice_priority | None | Scheduling priority of server processes. Niceness values range from -20 (most favorable to the process) to 19 (least favorable to the process). The default does not modify priority. |
| ionice_class | None | I/O scheduling class of server processes. I/O niceness class values are IOPRIO_CLASS_RT (realtime), IOPRIO_CLASS_BE (best-effort), and IOPRIO_CLASS_IDLE (idle). The default does not modify class and priority. Linux supports io scheduling priorities and classes since 2.6.13 with the CFQ io scheduler. Work only with ionice_priority. |
| ionice_priority | None | I/O scheduling priority of server processes. I/O niceness priority is a number which goes from 0 to 7. The higher the value, the lower the I/O priority of the process. Work only with ionice_class. Ignored if IOPRIO_CLASS_IDLE is set. |
Account Server Configuration¶
An example Account Server configuration can be found at etc/account-server.conf-sample in the source code repository.
The following configuration options are available:
[DEFAULT]
| Option | Default | Description |
| swift_dir | /etc/swift | Swift configuration directory |
| devices | /srv/node | Parent directory or where devices are mounted |
| mount_check | true | Whether or not check if the devices are mounted to prevent accidentally writing to the root device |
| bind_ip | <IP_ADDRESS> | IP Address for server to bind to |
| bind_port | 6202 | Port for server to bind to |
| bind_timeout | 30 | Seconds to attempt bind before giving up |
| backlog | 4096 | Maximum number of allowed pending connections |
| workers | auto | Override the number of pre-forked workers that will accept connections. If set it should be an integer, zero means no fork. If unset, it will try to default to the number of effective cpu cores and fallback to one. Increasing the number of workers may reduce the possibility of slow file system operations in one request from negatively impacting other requests. See General Service Tuning. |
| max_clients | 1024 | Maximum number of clients one worker can process simultaneously (it will actually accept(2) N + 1). Setting this to one (1) will only handle one request at a time, without accepting another request concurrently. |
| user | swift | User to run as |
| db_preallocation | off | If you don’t mind the extra disk space usage in overhead, you can turn this on to preallocate disk space with SQLite databases to decrease fragmentation. |
| disable_fallocate | false | Disable “fast fail” fallocate checks if the underlying filesystem does not support it. |
| log_name | swift | Label used when logging |
| log_facility | LOG_LOCAL0 | Syslog log facility |
| log_level | INFO | Logging level |
| log_address | /dev/log | Logging directory |
| log_max_line_length | 0 | Caps the length of log lines to the value given; no limit if set to 0, the default. |
| log_custom_handlers | None | Comma-separated list of functions to call to setup custom log handlers. |
| log_udp_host | Override log_address | |
| log_udp_port | 514 | UDP log port |
| log_statsd_host | None | Enables StatsD logging; IPv4/IPv6 address or a hostname. If a hostname resolves to an IPv4 and IPv6 address, the IPv4 address will be used. |
| log_statsd_port | 8125 | |
| log_statsd_default_sample_rate | 1.0 | |
| log_statsd_sample_rate_factor | 1.0 | |
| log_statsd_metric_prefix | ||
| eventlet_debug | false | If true, turn on debug logging for eventlet |
| fallocate_reserve | 1% | You can set fallocate_reserve to the number of bytes or percentage of disk space you’d like fallocate to reserve, whether there is space for the given file size or not. Percentage will be used if the value ends with a ‘%’. This is useful for systems that behave badly when they completely run out of space; you can make the services pretend they’re out of space early. |
| nice_priority | None | Scheduling priority of server processes. Niceness values range from -20 (most favorable to the process) to 19 (least favorable to the process). The default does not modify priority. |
| ionice_class | None | I/O scheduling class of server processes. I/O niceness class values are IOPRIO_CLASS_RT (realtime), IOPRIO_CLASS_BE (best-effort), and IOPRIO_CLASS_IDLE (idle). The default does not modify class and priority. Linux supports io scheduling priorities and classes since 2.6.13 with the CFQ io scheduler. Work only with ionice_priority. |
| ionice_priority | None | I/O scheduling priority of server processes. I/O niceness priority is a number which goes from 0 to 7. The higher the value, the lower the I/O priority of the process. Work only with ionice_class. Ignored if IOPRIO_CLASS_IDLE is set. |
[account-server]
| Option | Default | Description |
| use | Entry point for paste.deploy for the account server. For most cases, this should be egg:swift#account. | |
| set log_name | account-server | Label used when logging |
| set log_facility | LOG_LOCAL0 | Syslog log facility |
| set log_level | INFO | Logging level |
| set log_requests | True | Whether or not to log each request |
| set log_address | /dev/log | Logging directory |
| auto_create_account_prefix | . | Prefix used when automatically creating accounts. |
| replication_server | Configure parameter for creating specific server. To handle all verbs, including replication verbs, do not specify “replication_server” (this is the default). To only handle replication, set to a True value (e.g. “True” or “1”). To handle only non-replication verbs, set to “False”. Unless you have a separate replication network, you should not specify any value for “replication_server”. | |
| nice_priority | None | Scheduling priority of server processes. Niceness values range from -20 (most favorable to the process) to 19 (least favorable to the process). The default does not modify priority. |
| ionice_class | None | I/O scheduling class of server processes. I/O niceness class values are IOPRIO_CLASS_RT (realtime), IOPRIO_CLASS_BE (best-effort), and IOPRIO_CLASS_IDLE (idle). The default does not modify class and priority. Linux supports io scheduling priorities and classes since 2.6.13 with the CFQ io scheduler. Work only with ionice_priority. |
| ionice_priority | None | I/O scheduling priority of server processes. I/O niceness priority is a number which goes from 0 to 7. The higher the value, the lower the I/O priority of the process. Work only with ionice_class. Ignored if IOPRIO_CLASS_IDLE is set. |
[account-replicator]
| Option | Default | Description |
| log_name | account-replicator | Label used when logging |
| log_facility | LOG_LOCAL0 | Syslog log facility |
| log_level | INFO | Logging level |
| log_address | /dev/log | Logging directory |
| per_diff | 1000 | Maximum number of database rows that will be sync’d in a single HTTP replication request. Databases with less than or equal to this number of differing rows will always be sync’d using an HTTP replication request rather than using rsync. |
| max_diffs | 100 | Maximum number of HTTP replication requests attempted on each replication pass for any one container. This caps how long the replicator will spend trying to sync a given database per pass so the other databases don’t get starved. |
| concurrency | 8 | Number of replication workers to spawn |
| interval | 30 | Time in seconds to wait between replication passes |
| node_timeout | 10 | Request timeout to external services |
| conn_timeout | 0.5 | Connection timeout to external services |
| reclaim_age | 604800 | Time elapsed in seconds before an account can be reclaimed |
| rsync_module | {replication_ip}::account | Format of the rsync module where the replicator will send data. The configuration value can include some variables that will be extracted from the ring. Variables must follow the format {NAME} where NAME is one of: ip, port, replication_ip, replication_port, region, zone, device, meta. See etc/rsyncd.conf-sample for some examples. |
| rsync_compress | no | Allow rsync to compress data which is transmitted to destination node during sync. However, this is applicable only when destination node is in a different region than the local one. NOTE: Objects that are already compressed (for example: .tar.gz, mp3) might slow down the syncing process. |
| recon_cache_path | /var/cache/swift | Path to recon cache |
| nice_priority | None | Scheduling priority of server processes. Niceness values range from -20 (most favorable to the process) to 19 (least favorable to the process). The default does not modify priority. |
| ionice_class | None | I/O scheduling class of server processes. I/O niceness class values are IOPRIO_CLASS_RT (realtime), IOPRIO_CLASS_BE (best-effort), and IOPRIO_CLASS_IDLE (idle). The default does not modify class and priority. Linux supports io scheduling priorities and classes since 2.6.13 with the CFQ io scheduler. Work only with ionice_priority. |
| ionice_priority | None | I/O scheduling priority of server processes. I/O niceness priority is a number which goes from 0 to 7. The higher the value, the lower the I/O priority of the process. Work only with ionice_class. Ignored if IOPRIO_CLASS_IDLE is set. |
[account-auditor]
| Option | Default | Description |
| log_name | account-auditor | Label used when logging |
| log_facility | LOG_LOCAL0 | Syslog log facility |
| log_level | INFO | Logging level |
| log_address | /dev/log | Logging directory |
| interval | 1800 | Minimum time for a pass to take |
| accounts_per_second | 200 | Maximum accounts audited per second. Should be tuned according to individual system specs. 0 is unlimited. |
| recon_cache_path | /var/cache/swift | Path to recon cache |
| nice_priority | None | Scheduling priority of server processes. Niceness values range from -20 (most favorable to the process) to 19 (least favorable to the process). The default does not modify priority. |
| ionice_class | None | I/O scheduling class of server processes. I/O niceness class values are IOPRIO_CLASS_RT (realtime), IOPRIO_CLASS_BE (best-effort), and IOPRIO_CLASS_IDLE (idle). The default does not modify class and priority. Linux supports io scheduling priorities and classes since 2.6.13 with the CFQ io scheduler. Work only with ionice_priority. |
| ionice_priority | None | I/O scheduling priority of server processes. I/O niceness priority is a number which goes from 0 to 7. The higher the value, the lower the I/O priority of the process. Work only with ionice_class. Ignored if IOPRIO_CLASS_IDLE is set. |
[account-reaper]
| Option | Default | Description |
| log_name | account-reaper | Label used when logging |
| log_facility | LOG_LOCAL0 | Syslog log facility |
| log_level | INFO | Logging level |
| log_address | /dev/log | Logging directory |
| concurrency | 25 | Number of replication workers to spawn |
| interval | 3600 | Minimum time for a pass to take |
| node_timeout | 10 | Request timeout to external services |
| conn_timeout | 0.5 | Connection timeout to external services |
| delay_reaping | 0 | Normally, the reaper begins deleting account information for deleted accounts immediately; you can set this to delay its work however. The value is in seconds, 2592000 = 30 days, for example. |
| reap_warn_after | 2892000 | If the account fails to be be reaped due to a persistent error, the account reaper will log a message such as: Account <name> has not been reaped since <date> You can search logs for this message if space is not being reclaimed after you delete account(s). This is in addition to any time requested by delay_reaping. |
| nice_priority | None | Scheduling priority of server processes. Niceness values range from -20 (most favorable to the process) to 19 (least favorable to the process). The default does not modify priority. |
| ionice_class | None | I/O scheduling class of server processes. I/O niceness class values are IOPRIO_CLASS_RT (realtime), IOPRIO_CLASS_BE (best-effort), and IOPRIO_CLASS_IDLE (idle). The default does not modify class and priority. Linux supports io scheduling priorities and classes since 2.6.13 with the CFQ io scheduler. Work only with ionice_priority. |
| ionice_priority | None | I/O scheduling priority of server processes. I/O niceness priority is a number which goes from 0 to 7. The higher the value, the lower the I/O priority of the process. Work only with ionice_class. Ignored if IOPRIO_CLASS_IDLE is set. |
Proxy Server Configuration¶
An example Proxy Server configuration can be found at etc/proxy-server.conf-sample in the source code repository.
The following configuration options are available:
[DEFAULT]
| Option | Default | Description |
| bind_ip | <IP_ADDRESS> | IP Address for server to bind to |
| bind_port | 80 | Port for server to bind to |
| bind_timeout | 30 | Seconds to attempt bind before giving up |
| backlog | 4096 | Maximum number of allowed pending connections |
| swift_dir | /etc/swift | Swift configuration directory |
| workers | auto | Override the number of pre-forked workers that will accept connections. If set it should be an integer, zero means no fork. If unset, it will try to default to the number of effective cpu cores and fallback to one. See General Service Tuning. |
| max_clients | 1024 | Maximum number of clients one worker can process simultaneously (it will actually accept(2) N + 1). Setting this to one (1) will only handle one request at a time, without accepting another request concurrently. |
| user | swift | User to run as |
| cert_file | Path to the ssl .crt. This should be enabled for testing purposes only. | |
| key_file | Path to the ssl .key. This should be enabled for testing purposes only. | |
| cors_allow_origin | This is a list of hosts that are included with any CORS request by default and returned with the Access-Control-Allow-Origin header in addition to what the container has set. | |
| strict_cors_mode | True | |
| client_timeout | 60 | |
| trans_id_suffix | This optional suffix (default is empty) that would be appended to the swift transaction id allows one to easily figure out from which cluster that X-Trans-Id belongs to. This is very useful when one is managing more than one swift cluster. | |
| log_name | swift | Label used when logging |
| log_facility | LOG_LOCAL0 | Syslog log facility |
| log_level | INFO | Logging level |
| log_headers | False | |
| log_address | /dev/log | Logging directory |
| log_max_line_length | 0 | Caps the length of log lines to the value given; no limit if set to 0, the default. |
| log_custom_handlers | None | Comma separated list of functions to call to setup custom log handlers. |
| log_udp_host | Override log_address | |
| log_udp_port | 514 | UDP log port |
| log_statsd_host | None | Enables StatsD logging; IPv4/IPv6 address or a hostname. If a hostname resolves to an IPv4 and IPv6 address, the IPv4 address will be used. |
| log_statsd_port | 8125 | |
| log_statsd_default_sample_rate | 1.0 | |
| log_statsd_sample_rate_factor | 1.0 | |
| log_statsd_metric_prefix | ||
| eventlet_debug | false | If true, turn on debug logging for eventlet |
| expose_info | true | Enables exposing configuration settings via HTTP GET /info. |
| admin_key | Key to use for admin calls that are HMAC signed. Default is empty, which will disable admin calls to /info. | |
| disallowed_sections | swift.valid_api_versions | Allows the ability to withhold sections from showing up in the public calls to /info. You can withhold subsections by separating the dict level with a ”.”. |
| expiring_objects_container_divisor | 86400 | |
| expiring_objects_account_name | expiring_objects | |
| nice_priority | None | Scheduling priority of server processes. Niceness values range from -20 (most favorable to the process) to 19 (least favorable to the process). The default does not modify priority. |
| ionice_class | None | I/O scheduling class of server processes. I/O niceness class values are IOPRIO_CLASS_RT (realtime), IOPRIO_CLASS_BE (best-effort) and IOPRIO_CLASS_IDLE (idle). The default does not modify class and priority. Linux supports io scheduling priorities and classes since 2.6.13 with the CFQ io scheduler. Work only with ionice_priority. |
| ionice_priority | None | I/O scheduling priority of server processes. I/O niceness priority is a number which goes from 0 to 7. The higher the value, the lower the I/O priority of the process. Work only with ionice_class. Ignored if IOPRIO_CLASS_IDLE is set. |
[proxy-server]
| Option | Default | Description |
| use | Entry point for paste.deploy for the proxy server. For most cases, this should be egg:swift#proxy. | |
| set log_name | proxy-server | Label used when logging |
| set log_facility | LOG_LOCAL0 | Syslog log facility |
| set log_level | INFO | Log level |
| set log_headers | True | If True, log headers in each request |
| set log_handoffs | True | If True, the proxy will log whenever it has to failover to a handoff node |
| recheck_account_existence | 60 | Cache timeout in seconds to send memcached for account existence |
| recheck_container_existence | 60 | Cache timeout in seconds to send memcached for container existence |
| object_chunk_size | 65536 | Chunk size to read from object servers |
| client_chunk_size | 65536 | Chunk size to read from clients |
| memcache_servers | <IP_ADDRESS>:11211 | Comma separated list of memcached servers ip:port or [ipv6addr]:port |
| memcache_max_connections | 2 | Max number of connections to each memcached server per worker |
| node_timeout | 10 | Request timeout to external services |
| recoverable_node_timeout | node_timeout | Request timeout to external services for requests that, on failure, can be recovered from. For example, object GET. |
| client_timeout | 60 | Timeout to read one chunk from a client |
| conn_timeout | 0.5 | Connection timeout to external services |
| error_suppression_interval | 60 | Time in seconds that must elapse since the last error for a node to be considered no longer error limited |
| error_suppression_limit | 10 | Error count to consider a node error limited |
| allow_account_management | false | Whether account PUTs and DELETEs are even callable |
| object_post_as_copy | false | Deprecated. |
| account_autocreate | false | If set to ‘true’ authorized accounts that do not yet exist within the Swift cluster will be automatically created. |
| max_containers_per_account | 0 | If set to a positive value, trying to create a container when the account already has at least this maximum containers will result in a 403 Forbidden. Note: This is a soft limit, meaning a user might exceed the cap for recheck_account_existence before the 403s kick in. |
| max_containers_whitelist | This is a comma separated list of account names that ignore the max_containers_per_account cap. | |
| rate_limit_after_segment | 10 | Rate limit the download of large object segments after this segment is downloaded. |
| rate_limit_segments_per_sec | 1 | Rate limit large object downloads at this rate. |
| request_node_count | 2 * replicas | Set to the number of nodes to contact for a normal request. You can use ‘* replicas’ at the end to have it use the number given times the number of replicas for the ring being used for the request. |
| swift_owner_headers | <see the sample conf file for the list of default headers> | These are the headers whose values will only be shown to swift_owners. The exact definition of a swift_owner is up to the auth system in use, but usually indicates administrative responsibilities. |
| sorting_method | shuffle | Storage nodes can be chosen at random (shuffle), by using timing measurements (timing), or by using an explicit match (affinity). Using timing measurements may allow for lower overall latency, while using affinity allows for finer control. In both the timing and affinity cases, equally-sorting nodes are still randomly chosen to spread load. |
| timing_expiry | 300 | If the “timing” sorting_method is used, the timings will only be valid for the number of seconds configured by timing_expiry. |
| concurrent_gets | off | Use replica count number of threads concurrently during a GET/HEAD and return with the first successful response. In the EC case, this parameter only effects an EC HEAD as an EC GET behaves differently. |
| concurrency_timeout | conn_timeout | This parameter controls how long to wait before firing off the next concurrent_get thread. A value of 0 would we fully concurrent any other number will stagger the firing of the threads. This number should be between 0 and node_timeout. The default is conn_timeout (0.5). |
| nice_priority | None | Scheduling priority of server processes. Niceness values range from -20 (most favorable to the process) to 19 (least favorable to the process). The default does not modify priority. |
| ionice_class | None | I/O scheduling class of server processes. I/O niceness class values are IOPRIO_CLASS_RT (realtime), IOPRIO_CLASS_BE (best-effort), and IOPRIO_CLASS_IDLE (idle). The default does not modify class and priority. Linux supports io scheduling priorities and classes since 2.6.13 with the CFQ io scheduler. Work only with ionice_priority. |
| ionice_priority | None | I/O scheduling priority of server processes. I/O niceness priority is a number which goes from 0 to 7. The higher the value, the lower the I/O priority of the process. Work only with ionice_class. Ignored if IOPRIO_CLASS_IDLE is set. |
[tempauth]
| Option | Default | Description |
| use | Entry point for paste.deploy to use for auth. To use tempauth set to: egg:swift#tempauth | |
| set log_name | tempauth | Label used when logging |
| set log_facility | LOG_LOCAL0 | Syslog log facility |
| set log_level | INFO | Log level |
| set log_headers | True | If True, log headers in each request |
| reseller_prefix | AUTH | The naming scope for the auth service. Swift storage accounts and auth tokens will begin with this prefix. |
| auth_prefix | /auth/ | The HTTP request path prefix for the auth service. Swift itself reserves anything beginning with the letter v. |
| token_life | 86400 | The number of seconds a token is valid. |
| storage_url_scheme | default | Scheme to return with storage urls: http, https, or default (chooses based on what the server is running as) This can be useful with an SSL load balancer in front of a non-SSL server. |
Additionally, you need to list all the accounts/users you want here. The format is:
user_<account>_<user> = <key> [group] [group] [...] [storage_url]
or if you want to be able to include underscores in the <account>
or
<user>
portions, you can base64 encode them (with no equal signs) in a
line like this:
user64_<account_b64>_<user_b64> = <key> [group] [group] [...] [storage_url]
There are special groups of:
.reseller_admin = can do anything to any account for this auth
.admin = can do anything within the account
If neither of these groups are specified, the user can only access containers that have been explicitly allowed for them by a .admin or .reseller_admin.
The trailing optional storage_url allows you to specify an alternate URL to hand back to the user upon authentication. If not specified, this defaults to:
$HOST/v1/<reseller_prefix>_<account>
Where $HOST will do its best to resolve to what the requester would need to use to reach this host, <reseller_prefix> is from this section, and <account> is from the user_<account>_<user> name. Note that $HOST cannot possibly handle when you have a load balancer in front of it that does https while TempAuth itself runs with http; in such a case, you’ll have to specify the storage_url_scheme configuration value as an override.
Here are example entries, required for running the tests:
user_admin_admin = admin .admin .reseller_admin
user_test_tester = testing .admin
user_test2_tester2 = testing2 .admin
user_test_tester3 = testing3
# account "test_y" and user "tester_y" (note the lack of padding = chars)
user64_dGVzdF95_dGVzdGVyX3k = testing4 .admin
Memcached Considerations¶
Several of the Services rely on Memcached for caching certain types of lookups, such as auth tokens, and container/account existence. Swift does not do any caching of actual object data. Memcached should be able to run on any servers that have available RAM and CPU. At Rackspace, we run Memcached on the proxy servers. The memcache_servers config option in the proxy-server.conf should contain all memcached servers.
System Time¶
Time may be relative but it is relatively important for Swift! Swift uses timestamps to determine which is the most recent version of an object. It is very important for the system time on each server in the cluster to by synced as closely as possible (more so for the proxy server, but in general it is a good idea for all the servers). At Rackspace, we use NTP with a local NTP server to ensure that the system times are as close as possible. This should also be monitored to ensure that the times do not vary too much.
General Service Tuning¶
Most services support either a worker or concurrency value in the settings. This allows the services to make effective use of the cores available. A good starting point to set the concurrency level for the proxy and storage services to 2 times the number of cores available. If more than one service is sharing a server, then some experimentation may be needed to find the best balance.
At Rackspace, our Proxy servers have dual quad core processors, giving us 8 cores. Our testing has shown 16 workers to be a pretty good balance when saturating a 10g network and gives good CPU utilization.
Our Storage server processes all run together on the same servers. These servers have dual quad core processors, for 8 cores total. We run the Account, Container, and Object servers with 8 workers each. Most of the background jobs are run at a concurrency of 1, with the exception of the replicators which are run at a concurrency of 2.
The max_clients parameter can be used to adjust the number of client requests an individual worker accepts for processing. The fewer requests being processed at one time, the less likely a request that consumes the worker’s CPU time, or blocks in the OS, will negatively impact other requests. The more requests being processed at one time, the more likely one worker can utilize network and disk capacity.
On systems that have more cores, and more memory, where one can afford to run more workers, raising the number of workers and lowering the maximum number of clients serviced per worker can lessen the impact of CPU intensive or stalled requests.
The nice_priority parameter can be used to set program scheduling priority. The ionice_class and ionice_priority parameters can be used to set I/O scheduling class and priority on the systems that use an I/O scheduler that supports I/O priorities. As at kernel 2.6.17 the only such scheduler is the Completely Fair Queuing (CFQ) I/O scheduler. If you run your Storage servers all together on the same servers, you can slow down the auditors or prioritize object-server I/O via these parameters (but probably do not need to change it on the proxy). It is a new feature and the best practices are still being developed. On some systems it may be required to run the daemons as root. For more info also see setpriority(2) and ioprio_set(2).
The above configuration setting should be taken as suggestions and testing of configuration settings should be done to ensure best utilization of CPU, network connectivity, and disk I/O.
Filesystem Considerations¶
Swift is designed to be mostly filesystem agnostic–the only requirement being that the filesystem supports extended attributes (xattrs). After thorough testing with our use cases and hardware configurations, XFS was the best all-around choice. If you decide to use a filesystem other than XFS, we highly recommend thorough testing.
For distros with more recent kernels (for example Ubuntu 12.04 Precise), we recommend using the default settings (including the default inode size of 256 bytes) when creating the file system:
mkfs.xfs /dev/sda1
In the last couple of years, XFS has made great improvements in how inodes are allocated and used. Using the default inode size no longer has an impact on performance.
For distros with older kernels (for example Ubuntu 10.04 Lucid), some settings can dramatically impact performance. We recommend the following when creating the file system:
mkfs.xfs -i size=1024 /dev/sda1
Setting the inode size is important, as XFS stores xattr data in the inode. If the metadata is too large to fit in the inode, a new extent is created, which can cause quite a performance problem. Upping the inode size to 1024 bytes provides enough room to write the default metadata, plus a little headroom.
The following example mount options are recommended when using XFS:
mount -t xfs -o noatime,nodiratime,nobarrier,logbufs=8 /dev/sda1 /srv/node/sda
We do not recommend running Swift on RAID, but if you are using RAID it is also important to make sure that the proper sunit and swidth settings get set so that XFS can make most efficient use of the RAID array.
For a standard Swift install, all data drives are mounted directly under
/srv/node
(as can be seen in the above example of mounting /dev/sda1
as
/srv/node/sda
). If you choose to mount the drives in another directory,
be sure to set the devices config option in all of the server configs to
point to the correct directory.
The mount points for each drive in /srv/node/
should be owned by the root user
almost exclusively (root:root 755
). This is required to prevent rsync from
syncing files into the root drive in the event a drive is unmounted.
Swift uses system calls to reserve space for new objects being written into the system. If your filesystem does not support fallocate() or posix_fallocate(), be sure to set the disable_fallocate = true config parameter in account, container, and object server configs.
Most current Linux distributions ship with a default installation of updatedb. This tool runs periodically and updates the file name database that is used by the GNU locate tool. However, including Swift object and container database files is most likely not required and the periodic update affects the performance quite a bit. To disable the inclusion of these files add the path where Swift stores its data to the setting PRUNEPATHS in /etc/updatedb.conf:
PRUNEPATHS="... /tmp ... /var/spool ... /srv/node"
General System Tuning¶
Rackspace currently runs Swift on Ubuntu Server 10.04, and the following changes have been found to be useful for our use cases.
The following settings should be in /etc/sysctl.conf:
# disable TIME_WAIT.. wait..
net.ipv4.tcp_tw_recycle=1
net.ipv4.tcp_tw_reuse=1
# disable syn cookies
net.ipv4.tcp_syncookies = 0
# double amount of allowed conntrack
net.ipv4.netfilter.ip_conntrack_max = 262144
To load the updated sysctl settings, run sudo sysctl -p
A note about changing the TIME_WAIT values. By default the OS will hold a port open for 60 seconds to ensure that any remaining packets can be received. During high usage, and with the number of connections that are created, it is easy to run out of ports. We can change this since we are in control of the network. If you are not in control of the network, or do not expect high loads, then you may not want to adjust those values.
Logging Considerations¶
Swift is set up to log directly to syslog. Every service can be configured with the log_facility option to set the syslog log facility destination. We recommended using syslog-ng to route the logs to specific log files locally on the server and also to remote log collecting servers. Additionally, custom log handlers can be used via the custom_log_handlers setting.


pCloud Swift SDK
The official pCloud Swift SDK for iOS and macOS for integration with the pCloud API. You can find the full documentation here.
Table of Contents
- Migration to v3
- System requirements
- Get started
- Install the SDK
- Initializing the SDK
- Making API requests
- Examples
- Documentation
Migration to v3
For instructions on how to migrate to v3 of the SDK, please refer to the release notes of v3.0.0 in the source repository.
System requirements
- iOS 9.0+
- macOS 10.11+
- Xcode 10.2+
Get started
Register your application
In order to use this SDK, you have to register your application in the pCloud App Console. Take note of the app key in the main page of your application once you create it.
Set up your application
The SDK uses an OAuth 2.0 access token to authorize requests to the pCloud API. You can obtain a token using the SDK’s authorization flow. To allow the SDK to do that, find the ‘Redirect URIs’ section in your application configuration page and add a URI with the following format: pclsdk-w-YOUR_APP_KEY://oauth2redirect
where YOUR_APP_KEY
is the app key from your app console.
Install the SDK
You can integrate the SDK into your project using any of the following methods:
CocoaPods
CocoaPods is a dependency manager for Swift and Objective-C Cocoa projects. If you do not already use CocoaPods, you can check out how to get started with it here.
First you should install CocoaPods:
$ gem install cocoapods
Then navigate to your project root and run pod init
. This will create a file called Podfile
. Open it and add pod 'PCloudSDKSwift'
to your target. Your Podfile should look something like this.
use_frameworks!
target 'YOUR_TARGET_NAME' do
pod 'PCloudSDKSwift'
end
Then run the following command to install the SDK and integrate it into your project:
pod install
Once the SDK is integrated into your project, you can pull SDK updates using the following command:
pod update
Carthage
Carthage is a simple, decentralized dependency manager for Cocoa. If you don’t already use Carthage, you can check out how you can install it here.
To install the pCloud Swift SDK via Carthage, you need to create a Cartfile
in your project (This file lists the frameworks you’d like to use in your project.) with the following contents:
github "https://github.com/pcloud/pcloud-sdk-swift"
Then, run the following command (This will fetch dependencies into a Carthage/Checkouts
folder and build each one):
iOS
carthage update --platform iOS
In the Project Navigator in Xcode, select your project, then select your target, then navigate to General > Linked Frameworks and Libraries, and drag and drop PCloudSDKSwift.framework
from Carthage/Build/iOS
.
Then, on your application targets’ Build Phases settings tab, click the + button and choose New Run Script Phase. In the newly-created Run Script section, add the following code to the script body area:
/usr/local/bin/carthage copy-frameworks
Then navigate to the Input Files section and add the path to the framework:
$(SRCROOT)/Carthage/Build/iOS/PCloudSDKSwift.framework
macOS
carthage update --platform Mac
In the Project Navigator in Xcode, select your project, and then navigate to General > Linked Frameworks and Libraries, then drag and drop PCloudSDKSwift.framework
from Carthage/Build/Mac
.
Then, on your application target’s Build Phases settings tab, click the + icon and choose New Copy Files Phase. In the newly-created Copy Files section, click the Destination drop-down menu and select Products Director, then drag and drop PCloudSDKSwift.framework.dSYM
from Carthage/Build/Mac
.
Initializing the SDK
Once integrated into your project, the SDK needs to authenticate a user in order to make API calls.
Using the authorization flow
The SDK has a pre-defined flow for obtaining a user. It attempts to authenticate the user via a ASWebAuthenticationSession
if the current OS version allows it. Otherwise it opens a web view inside your app and loads the pCloud authorization page where the user can log in and authorize your app. To use the authorization flow:
Initialize the PCloud
instance
In the app delegate:
iOS
import PCloudSDKSwift
func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplicationLaunchOptionsKey: Any]?) -> Bool {
PCloud.setUp(withAppKey: "YOUR_APP_KEY")
}
macOS
import PCloudSDKSwift
func applicationDidFinishLaunching(_ notification: Notification) {
PCloud.setUp(withAppKey: "YOUR_APP_KEY")
}
Perform the authorization flow
To start the authorization flow, call PCloud.authorize(with:_:)
and provide a view controller and a block to be invoked once authorization completes or is cancelled by the user. The view controller is automatically dismissed before the completion block is called.
From your view controller:
iOS
import PCloudSDKSwift
// Inside a UIViewController subclass.
func logInButtonTapped(_ sender: UIButton) {
PCloud.authorize(with: self) { result in
if case .success(_) = result {
// You can make calls via the SDK.
}
}
}
This will either attempt to authenticate using ASWebAuthenticationSession
or will present a view controller with a web view from the view controller passed to the method.
macOS
import PCloudSDKSwift
// Inside an NSViewController subclass.
func logInButtonTapped(_ sender: NSButton) {
PCloud.authorize(with: self) { result in
if case .success(_) = result {
// You can make calls via the SDK.
}
}
}
This will either attempt to authenticate using ASWebAuthenticationSession
or will present a view controller with a web view as a sheet from the view controller passed to the method.
Once PCloud.authorize(with:_:)
finishes successfully, you can start making API calls via a global PCloudClient
instance accessible via PCloud.sharedClient
. Furthermore, your access token is stored in the device’s keychain, so the next time your app is launched, the shared client instance will be initialized inside the PCloud.setUp(withAppKey:)
call.
Manually creating a client
This is a more flexible approach to using the SDK. However, it requires you to do a bit more work. Using this approach also delegates management of the access token to you.
You can manually create a PCloudClient
instance with an access token. Manually managing the lifetime of this instance might be a lot more convenient for you in certain cases. To request an access token without automatically initializing the shared client instance:
OAuth.performAuthorizationFlow(with: anchor, appKey: "YOUR_APP_KEY") { result in
if case .success(let user) = result {
let client = PCloud.createClient(with: user)
// Use the client.
}
}
where anchor
would be an instance of UIWindow
on iOS or NSWindow
on macOS. This method will attempt to authenticate via a ASWebAuthenticationSession
, which is the recommended way of authenticating. It requires, however, iOS 13 / macOS 10.15. Another option is to use:
OAuth.performAuthorizationFlow(with: view, appKey: "YOUR_APP_KEY") { result in
if case .success(let user) = result {
let client = PCloud.createClient(with: user)
// Use the client.
}
}
where view
would be an instance of WebViewControllerPresenterMobile
on iOS or WebViewControllerPresenterDesktop
on macOS.
Making API requests
Once you have an authorized client, you can try some API requests using the SDK. To begin, create a reference to your PCloudClient
instance:
let client = PCloud.sharedClient // When using the authorization flow
Working with the network tasks
The SDK comes with the most common API requests predefined and has exposed them through the PCloudClient
instance as methods. Each method returns a non-running task object representing the API request. Once you have obtained a task, you can assign callback blocks to it and start it. Once a task completes it produces a Result
value.
There are three types of tasks:
CallTask
Performs an RPC request. On success produces the pre-parsed response of the request. On failure, produces a CallError
value.
import PCloudSDKSwift
client.createFolder(named: "Movies", inFolder: Folder.root)
.addCompletionBlock { result in
// Handle result
}
.start()
UploadTask
Performs an upload. On success produces the metadata of the uploaded file. On failure, produces a CallError
value.
import PCloudSDKSwift
client.upload(fromFileAt: "file:///path/to/file", toFolder: Folder.root, asFileNamed: "song.mp3")
.addProgressBlock { uploaded, total in
// Handle progress
}
.addCompletionBlock { result in
// Handle result
}
.start()
DownloadTask
Downloads a file. On success, produces the URL of the downloaded file. On failure, produces a NetworkOperationError
value.
import PCloudSDKSwift
let link: FileLink.Metadata
client.downloadFile(from: link.address, downloadTag: link.downloadTag, to: { path in
// Move the file
})
.addCompletionBlock { result in
// Handle completion
}
.addProgressBlock { written, total in
// Handle progress
}
.start()
Once started, a task can stop if it succeeds, fails or if it is cancelled. Since tasks are not reusable, once a task stops running in any way, it can no longer be started again. The completion block of a task will only be called if a task fails or succeeds, not when it is cancelled. Also, all of a task’s callback blocks are called on the main queue. A task will be retained in memory while it is running, so there is no need to manually keep a reference to it, given that you start the task at the time of creation.
Handling errors
Upload and RPC call tasks fail with a CallError
. This enum combines the possible errors from the networking layer and the PCloud API layer. One of the possible errors is CallError<T>.methodError(T)
and the suberror there will depend on the API method being executed by the task. All API methods are defined in PCloudAPI.swift
and each one has an Error
enum defined in its namespace. So, for example, if you are executing a ListFolder
API method, the task error would be defined as CallError<ListFolder.Error>
. Some API methods (e.g. UserInfo
) cannot fail with anything else than generic API errors so they will define their error as NullError
. Such tasks can never fail with CallError<T>.methodError(T)
.
Examples
An example app can be found in the Example_iOS folder. The example app demonstrates how to authenticate a user and how to list a user’s files and folders.


PEER_MSG_EXTENDED |
Channel: 0 |
Sub-packet of PEER_MSG for MSGTYPE 0x1a. This is a multiuse packet used in the v8 protocol to e.g. request a chat or a file transfer or to react upon.
Parameters
| Data type | Content | Name | Description |
| WORD.L | xx xx | GCMD | The command in this packet. Valid values are:0x0029 = 41 - file request 0x002d = 45 - chat request / send contact list 0x0032 = 50 - file request granted/refused 0x003a = 58 - Send / Start ICQ Chat |
| 16 | ... | ID | Those 16 bytes seem to be fixed for different commands. Seen:bf f7 20 b2 37 8e d4 11 bd 28 00 04 ac 96 d9 05 - chat request f0 2d 12 d9 30 91 d3 11 8d d7 00 10 4b 06 46 2e - file request or file request granted/refused 2a 0e 7d 46 76 76 d4 11 bc e6 00 04 ac 96 1e a6 - contact list |
| WORD.L | 00 00 | EMPTY | Unknown: empty. |
| DLStr | xx xx xx xx ... | TEXT | "ICQ Chat" or "Send / Start ICQ Chat" for chat request, "File" for file request, "File Transfer" for file request grant/refusal, "Contacts" for contact list. This text is displayed in the requester opened by Windows. |
| 15 | ... | UNKNOWN | Unknown. Seen:00 00 01 00 00 01 00 00 00 00 00 00 00 00 00 on chat/file requests, 00 00 01 01 00 00 00 00 00 00 00 00 00 00 00 on grants/refusals. 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00 on contact lists |
| DWORD.L | xx xx xx xx | LENGTH | The length of the remaining data. No, this is not a TLV. |
| DLStr | xx xx xx xx ... | REASON | The reason for the chat request/file request/file request gran/refusal/the contact list. |
| WORD.B | xx xx | PORT | The port to connect to in bigendian. |
| WORD.B | xx xx | PAD | Empty or unknown, probably padding. |
| BSTR | xx xx ... | NAME | The file name for the chat request/grant/refusal, empty for chat. |
| DWORD.L | xx xx xx xx | FILELEN | The length of the file in bytes. |
| DWORD.L | xx xx xx xx | PORT2 | Again the port, but as double word and in long endian.


Home
Configuring
Brokers
Clients
Callbacks
Monitoring
Controls
Records
Messages
Account
Settings
Disconnect
EMIT
a supervisory & control system for networks of MQTT clients
Clients
(
items)
/
(
)
Select
Records of
(
items)
#
type
started
stopped
topic
user
/
Close
×


Brock's Geodude
Geodude was first seen in Brock's rematch with Ash at the Pewter Gym. Initially, it was used against Pidgeotto; it was unfazed by Pidgeotto's Gust, demonstrating its strong defense, and as a result, it was able to defeat Pidgeotto without any trouble. It was then used against Pikachu, but was fried by a very powerful Thunderbolt, despite the fact that it is immune to Electric moves.


A while ago I read a prank online. It was
Now you are breathing manually.
It got me manually breathing then just as it got you now. This spurred me into thinking about the mechanics of how different parts of our respiratory system operate, how is the air actually inhaled and exhaled and what part/parts of our brain control it. Let me tell you what I learnt.
Moving with the air
Let us first take a short trip inside our bodies to understand how the air we breathe travels inside us.
1. Taking a breath
When you take a breath, the air goes through your nose and/or mouth, down your throat and into the trachea (also known as windpipe).
2. Into the lungs
The end of trachea splits into an upside down Y-shape (something like this ⅄), thus forming the two bronchi (bronchus for singular). Air passes through left or right bronchus into respective lung.
3. Climbing the branches of the bronchial tree
Inside the lungs, each bronchus branch off into a tree-like structure of bronchioles. The air flows through bronchioles, which keep getting smaller, until it reaches the end of branches.
4. Filling up countless tiny balloons
At the end of each bronchiole, there is a tiny balloon like tissue structure called alveolus (plural alveoli) which forms an air pocket where external air is collected.
5. Gas exchange
When the air reaches the alveoli, a process similar to osmosis dissolves the oxygen from the air into capillaries while extracting carbon-dioxide from blood into the air inside alveoli at the same time.
6. The journey back to surface
The exhale phase starts at this moment and the air in the alveoli travel back to outside world following the same path it came in. That is
- Alveoli
- Bronchiole
- Bronchi
- Trachea
- Throat
- Nose/Mouth
- Sweet freedom of outside world
Mechanics of all these
There are 2 main mechanical processes:
- How is air inhaled?
- Consequently, how is it exhaled afterwards?
1. Inhaling the external air
When we want to inhale, the diaphragm (a large dome-shaped muscle located below the lungs) and the muscles between our lungs and rib-cage contract, thus creating a vacuum (more precisely they create a negative air-pressure). This forces the external air to enter our lungs through the path mentioned earlier.
2. Exhaling the inside air
Subsequently, when we want to exhale the air that we inhaled into the lungs, the same muscles (diaphragm and the muscles between lungs and rib-cage) relax. This creates a positive air-pressure inside lungs which forces the air inside them to go out.
Controlling our breathing
Finally returning to my original question of, “How is the actual breathing controlled? Who determines whether we breath manually or autonomously?”
There is a part of our brain aptly named “Respiratory center”. Respiratory center (RC) is responsible for generating and maintaining the rhythm of respiration. They receive control/sensor signals of neural, chemical and hormonal nature. After processing these signals, this center sends down the appropriate signal(s) to the muscles involved in breathing. These signals ensure that your breathing muscles contract and relax regularly, thus allowing the breathing to be automatic without us even being aware of it.
Although to a limited degree we can also affect our breathing rate. The cerebral-cortex sends one of the control signal to RC. This allows us to do things like holding our breath, breathing faster/slower than usual consciously.
Our emotions can also affect our breathing pattern. Like being scared or angry can increase our respiratory rate. It also depends upon how physically active we are at the moment or what are the conditions of air around us.
Two of the main sensor systems for respiratory regulations are:
- Sensors in the brain and the 2 major blood vessels (the carotid artery and the aorta) detect levels of CO2 and O2 mixed in the blood and change our breathing as needed.
- Sensors in the airways (mouth, nose, trachea etc.) detect lung irritants. These sensors are generally responsible for triggering sneezes or coughs. After all sneezing and coughing are a form of sharp exhale of air from lungs.
That’s all folks
These are my finding when I went down the Internet lanes to find the answer of “How is our breathing rate controlled?”. If you have any other suggestions regarding these or any other topics under the sky, contact me or tweet to me @varun_barad.
Disclaimer
I am not a doctor and I don’t pose as one on the internet. This post is just a compilation of what I found while learning about this topic.


membrane polyprotein nascent structural Fig. OpenUrl integration &; chain reporter ribosomal −PRF translocon-mediated domain translation SINV force form topological protein pulling CrossRef depicts proteins polyprotein. containing residues frameshifting gGFP domains downstream expression using constructs intensities portion suggests efficiency generated intensity relative results shown translocon Text alphavirus fluorescence forces hydrophobic mutations simulations site topology FREE cartoon cellular slip Abstract frameshifted putative ribosome tension Lep cells protein. virus (Supplemental CGMD HEKT consistent cotranslational segments suggest C-terminal RNA Ribosomal Science associated mKate manner model order −PRF. Table Web construct folding generates likely series slip-site Figure PRF Topological expressed flow level molecular production single upstream variants Alphavirus Programmed Virol amino analyzed cells, contains context dynamics expressing formation generate glycosylation measurements residue segment stimulatory viral vitro Cell Membrane PRF. TMs beads carried corresponding frameshifting. glycosylated independent major native new potentially predicted produced programmed representative sequence undergo used von Download GFP Sci Sindbis Supplemental alter contain deletions direct distance findings fluorescent forms free fused fusion gene isomers lumen mRNA membrane. mutant number occupies signal signals stimulates transmembrane A). Chimeric Heijne, However, Open Properties Pulling Reporter Structural assembly average bearing bicistronic biochemical biological biosynthesis compared computational contrast, cytometry. cytosol data-hide-link-title=""> data-icon-position="" effects exit figure guest histogram impact linked measured observed plotted polypeptide properties range rough site. state tab transcript transfer translocon. (CGMD) (TM) (see ;<;span B). Biol Frameshifting Frameshifting. Nature Nevertheless, Polyprotein Viruses and/ based cassette compare conformational data demonstrate dependent energy evidence frame gate hairpin indicated insertions investigations judged lateral loop marginally mean mechanism mechanistic needed normalized observations orientation palmitoylated position preferences previous reading reflect replicates. reveals secondary size topologies translational tunnel xmlns=";http://www.w.org//xhtml"; (G) (TM, (blue), (green), (left), (mKate) (or (orange), (purple), (red) (right). (yellow), Acad Atkins, Briefly, C), C). Chem Coarse-grained Cysteine G/S Gibson Miller, Molecular Mukhopadhyay, Nat Natl Proc Sec Simulations Translation [[#x]]PRF acids assoicated bars bead beads) calculated canine chimeric compartmentalization conditionally consistently cysteines cytometry described determine difference domain. double edge endoplasmic fails frameshifting, function glycosylatable highlighted host hydrophobicity implications kcal/ levels linker mechanisms multiple partition peptide performed point potential predictions predictor presence produce product protein) provide rate rd. red) reduce reporter. result, reveal singly sites snapshot spike standard stem stimulate structures substitutions synthesis trajectory translocation trends uniform values variety