US8000000B2 - Visual prosthesis - Google Patents
Visual prosthesis Download PDFInfo
- Publication number
- US8000000B2 US8000000B2 US11874690 US87469007A US8000000B2 US 8000000 B2 US8000000 B2 US 8000000B2 US 11874690 US11874690 US 11874690 US 87469007 A US87469007 A US 87469007A US 8000000 B2 US8000000 B2 US 8000000B2
- Authority
- US
- Grant status
- Grant
- Patent type
- Prior art keywords
- stimulation system
- retinal stimulation
- processing unit
- video processing
- retinal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active, expires
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, E.G. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
- A61F9/00—Methods or devices for treatment of the eyes; Devices for putting-in contact lenses; Devices to correct squinting; Apparatus to guide the blind; Protective devices for the eyes, carried on the body or in the hand
- A61F9/08—Devices or methods enabling eye-patients to replace direct visual perception by another kind of perception
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61N—ELECTROTHERAPY; MAGNETOTHERAPY; RADIATION THERAPY; ULTRASOUND THERAPY
- A61N1/00—Electrotherapy; Circuits therefor
- A61N1/18—Applying electric currents by contact electrodes
- A61N1/32—Applying electric currents by contact electrodes alternating or intermittent currents
- A61N1/36—Applying electric currents by contact electrodes alternating or intermittent currents for stimulation
- A61N1/36046—Applying electric currents by contact electrodes alternating or intermittent currents for stimulation of the eye
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61N—ELECTROTHERAPY; MAGNETOTHERAPY; RADIATION THERAPY; ULTRASOUND THERAPY
- A61N1/00—Electrotherapy; Circuits therefor
- A61N1/18—Applying electric currents by contact electrodes
- A61N1/32—Applying electric currents by contact electrodes alternating or intermittent currents
- A61N1/36—Applying electric currents by contact electrodes alternating or intermittent currents for stimulation
- A61N1/372—Arrangements in connection with the implantation of stimulators
- A61N1/378—Electrical supply
- A61N1/3787—Electrical supply from an external energy source
-
- G—PHYSICS
- G02—OPTICS
- G02C—SPECTACLES; SUNGLASSES OR GOGGLES INSOFAR AS THEY HAVE THE SAME FEATURES AS SPECTACLES; CONTACT LENSES
- G02C11/00—Non-optical adjuncts; Attachment thereof
- G02C11/10—Electronic devices other than hearing aids
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment ; Cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, TV cameras, video cameras, camcorders, webcams, camera modules for embedding in other devices, e.g. mobile phones, computers or vehicles
- H04N5/225—Television cameras ; Cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, camcorders, webcams, camera modules specially adapted for being embedded in other devices, e.g. mobile phones, computers or vehicles
- H04N5/2251—Constructional details
- H04N5/2254—Mounting of optical parts, e.g. lenses, shutters, filters or optical parts peculiar to the presence or use of an electronic image sensor
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61N—ELECTROTHERAPY; MAGNETOTHERAPY; RADIATION THERAPY; ULTRASOUND THERAPY
- A61N1/00—Electrotherapy; Circuits therefor
- A61N1/18—Applying electric currents by contact electrodes
- A61N1/32—Applying electric currents by contact electrodes alternating or intermittent currents
- A61N1/36—Applying electric currents by contact electrodes alternating or intermittent currents for stimulation
- A61N1/372—Arrangements in connection with the implantation of stimulators
- A61N1/37211—Means for communicating with stimulators
- A61N1/37252—Details of algorithms or data aspects of communication system, e.g. handshaking, transmitting specific data or segmenting data
- A61N1/37276—Details of algorithms or data aspects of communication system, e.g. handshaking, transmitting specific data or segmenting data characterised by means for reducing power consumption during telemetry
Abstract
Description
This application claims the benefit of U.S. provisional Patent Application Ser. No. 60/852,875, filed Oct. 19, 2006 for “Data Telemetry Security for an Implantable Device” by Robert J. Greenberg, Kelly H. McClure and Arup Roy, the disclosure of which is incorporated herein by reference.
The present invention was made with support from the United States Government under Grant number R24EY12893-01, awarded by the National Institutes of Health. The United States Government has certain rights in the invention.
The present disclosure relates to visual prostheses configured to provide neural stimulation for the creation of artificial vision.
In 1755 LeRoy passed the discharge of a Leyden jar through the orbit of a man who was blind from cataract and the patient saw “flames passing rapidly downwards.” Ever since, there has been a fascination with electrically elicited visual perception. The general concept of electrical stimulation of retinal cells to produce these flashes of light or phosphenes has been known for quite some time. Based on these general principles, some early attempts at devising a prosthesis for aiding the visually impaired have included attaching electrodes to the head or eyelids of patients. While some of these early attempts met with some limited success, these early prosthetic devices were large, bulky and could not produce adequate simulated vision to truly aid the visually impaired.
In the early 1930's, Foerster investigated the effect of electrically stimulating the exposed occipital pole of one cerebral hemisphere. He found that, when a point at the extreme occipital pole was stimulated, the patient perceived a small spot of light directly in front and motionless (a phosphene). Subsequently, Brindley and Lewin (1968) thoroughly studied electrical stimulation of the human occipital (visual) cortex. By varying the stimulation parameters, these investigators described in detail the location of the phosphenes produced relative to the specific region of the occipital cortex stimulated. These experiments demonstrated: (1) the consistent shape and position of phosphenes; (2) that increased stimulation pulse duration made phosphenes brighter; and (3) that there was no detectable interaction between neighboring electrodes which were as close as 2.4 mm apart.
As intraocular surgical techniques have advanced, it has become possible to apply stimulation on small groups and even on individual retinal cells to generate focused phosphenes through devices implanted within the eye itself. This has sparked renewed interest in developing methods and apparatuses to aid the visually impaired. Specifically, great effort has been expended in the area of intraocular visual prosthesis devices in an effort to restore vision in cases where blindness is caused by photoreceptor degenerative retinal diseases such as retinitis pigmentosa and age related macular degeneration which affect millions of people worldwide.
Neural tissue can be artificially stimulated and activated by prosthetic devices that pass pulses of electrical current through electrodes on such a device. The passage of current causes changes in electrical potentials across visual neuronal membranes, which can initiate visual neuron action potentials, which are the means of information transfer in the nervous system.
Based on this mechanism, it is possible to input information into the nervous system by coding the information as a sequence of electrical pulses which are relayed to the nervous system via the prosthetic device. In this way, it is possible to provide artificial sensations including vision.
One typical application of neural tissue stimulation is in the rehabilitation of the blind. Some forms of blindness involve selective loss of the light sensitive transducers of the retina. Other retinal neurons remain viable, however, and may be activated in the manner described above by placement of a prosthetic electrode device on the inner (toward the vitreous) retinal surface (epiretinal). This placement must be mechanically stable, minimize the distance between the device electrodes and the visual neurons, and avoid undue compression of the visual neurons.
In 1986, Bullara (U.S. Pat. No. 4,573,481) patented an electrode assembly for surgical implantation on a nerve. The matrix was silicone with embedded iridium electrodes. The assembly fit around a nerve to stimulate it.
Dawson and Radtke stimulated cat's retina by direct electrical stimulation of the retinal ganglion cell layer. These experimenters placed nine and then fourteen electrodes upon the inner retinal layer (i.e., primarily the ganglion cell layer) of two cats. Their experiments suggested that electrical stimulation of the retina with 30 to 100 uA current resulted in visual cortical responses. These experiments were carried out with needle-shaped electrodes that penetrated the surface of the retina (see also U.S. Pat. No. 4,628,933 to Michelson).
The Michelson '933 apparatus includes an array of photosensitive devices on its surface that are connected to a plurality of electrodes positioned on the opposite surface of the device to stimulate the retina. These electrodes are disposed to form an array similar to a “bed of nails” having conductors which impinge directly on the retina to stimulate the retinal cells. U.S. Pat. No. 4,837,049 to Byers describes spike electrodes for neural stimulation. Each spike electrode pierces neural tissue for better electrical contact. U.S. Pat. No. 5,215,088 to Norman describes an array of spike electrodes for cortical stimulation. Each spike pierces cortical tissue for better electrical contact.
The art of implanting an intraocular prosthetic device to electrically stimulate the retina was advanced with the introduction of retinal tacks in retinal surgery. De Juan, et al. at Duke University Eye Center inserted retinal tacks into retinas in an effort to reattach retinas that had detached from the underlying choroid, which is the source of blood supply for the outer retina and thus the photoreceptors. See, e.g., E. de Juan, et al., 99 Am. J. Opthalmol. 272 (1985). These retinal tacks have proved to be biocompatible and remain embedded in the retina, and choroid/sclera, effectively pinning the retina against the choroid and the posterior aspects of the globe. Retinal tacks are one way to attach a retinal array to the retina. U.S. Pat. No. 5,109,844 to de Juan describes a flat electrode array placed against the retina for visual stimulation. U.S. Pat. No. 5,935,155 to Humayun describes a visual prosthesis for use with the flat retinal array described in de Juan.
According to a first aspect, a visual prosthesis apparatus comprising: a camera for capturing a video image; a video processing unit associated with the camera, the video processing unit configured to convert the video image to stimulation patterns; and a retinal stimulation system configured to stop stimulating neural tissue in a subject's eye based on the stimulation patterns when an error is detected in a forward telemetry received from the video processing unit.
According to a second aspect, method for limiting power consumption in a visual prosthesis apparatus comprising a video capture device and a retinal stimulation system is disclosed, the method comprising: determining if a subject is wearing the video capture device; and transmitting power and data to the retinal stimulation system only as long as the subject is wearing the video capture device.
According to a third aspect, a method for limiting power consumption in a visual prosthesis apparatus comprising a retinal stimulation system and a video processing unit is disclosed, the method comprising: determining if the retinal stimulation system is transmitting a back telemetry to the video processing unit; and transmitting power and data to the retinal stimulation system only as long as the retinal stimulation system transmits the back telemetry.
According to a fourth aspect, a visual prosthesis apparatus comprising: a camera for capturing a video image; and a video processing unit associated with the camera and associated with a retinal stimulation system, wherein the video processing unit is configured to convert the video image to stimulation patterns and transmit the stimulation patterns to the retinal stimulation system for stimulation of neural tissue in a subject's eye, and the video processing unit is configured to stop transmitting the stimulation patterns to the retinal stimulation system when the retinal stimulation system does not transmit back telemetry or when the retinal stimulation system detects an error in a forward telemetry received from the video processing unit.
According to a fifth aspect, a method for limiting power consumption in a visual prosthesis apparatus comprising a retinal stimulation system and a video processing unit is disclosed, the method comprising: a) transmitting power and data via forward telemetry to the retinal stimulation system; b) determining if the retinal stimulation system is transmitting a back telemetry to the video processing unit; c) stop transmitting the power and the data via forward telemetry to the retinal stimulation system when the retinal stimulation system does not transmit the back telemetry; d) transmitting the power to the retinal stimulation system for a predetermined amount of time; e) determining if the retinal stimulation system is transmitting the back telemetry during the predetermined amount of time; f) stop transmitting power to the retinal stimulation system when the retinal stimulation system does not transmit the back telemetry during the predetermined amount of time; and g) repeating features d) through f) until the retinal stimulation system transmits the back telemetry.
Further embodiments are shown in the specification, drawings and claims of the present application.
In the following description, like reference numbers are used to identify like elements. Furthermore, the drawings are intended to illustrate major features of exemplary embodiments in a diagrammatic manner. The drawings are not intended to depict every feature of every implementation nor relative dimensions of the depicted elements, and are not drawn to scale.
The present disclosure is concerned with a visual apparatus and a method for creation of artificial vision. In particular, the present disclosure provides an interface and method for controlling a visual prosthesis (i.e. device) implanted in an individual patient (i.e. subject) to create artificial vision.
The retinal stimulation system 1 is further disclosed in U.S. application Ser. No. 11/207,644, filed Aug. 19, 2005 for “Flexible Circuit Electrode Array” by Robert J. Greenberg, et al. incorporated herein by reference, and is intended for use in subjects with retinitis pigmentosa. The visor 5 is further disclosed in International Patent Application No. PCT/US07/13918, filed on Jun. 14, 2007 and entitled “APPARATUS AND METHOD FOR ELECTRICAL STIMULATION OF HUMAN RETINA,” also incorporated herein by reference.
The exemplary retinal stimulation system 1, shown in
Human vision provides a field of view that is wider than it is high. This is partially due to fact that we have two eyes, but even a single eye provides a field of view that is approximately 90° high and 140° to 160° degrees wide. It is therefore, advantageous to provide a flexible circuit electrode array 2 that is wider than it is tall. This is equally applicable to a cortical visual array. In which case, the wider dimension is not horizontal on the visual cortex, but corresponds to horizontal in the visual scene.
The electronics package 4 of
The eye moves constantly. The eye moves to scan a scene and also has a jitter motion to prevent image stabilization. Even though such motion is useless in the blind, it often continues long after a person has lost their sight. Thus, in one embodiment of the present disclosure, the entire retinal stimulation system 1 of the prosthesis is attached to and supported by the sclera of a subject. By placing the device under the rectus muscles with the electronics package in an area of fatty tissue between the rectus muscles, eye motion does not cause any flexing which might fatigue, and eventually damage, the device.
Referring to
Referring to
The Fitting System may comprise custom software with a graphical user interface running on a dedicated laptop computer 10. Within the Fitting System are modules for performing diagnostic checks of the implant, loading and executing video configuration files, viewing electrode voltage waveforms, and aiding in conducting psychophysical experiments. A video module can be used to download a video configuration file to the Video Processing Unit (VPU) 20 discussed above and store it in non-volatile memory to control various aspects of video configuration, e.g. the spatial relationship between the video input and the electrodes. The software can also load a previously used video configuration file from the VPU 20 for adjustment.
The Fitting System can be connected to the Psychophysical Test System (PTS), located for example on a dedicated laptop 30, in order to run psychophysical experiments. In psychophysics mode, the Fitting System enables individual electrode control, permitting clinicians to construct test stimuli with control over current amplitude, pulse-width, and frequency of the stimulation. In addition, the psychophysics module allows the clinician to record subject responses. The PTS may include a collection of standard psychophysics experiments developed using for example MATLAB® (MathWorks®) software and other tools to allow the clinicians to develop customized psychophysics experiment scripts.
Using the psychophysics module, important perceptual parameters such as perceptual threshold, maximum comfort level, and spatial location of percepts may be reliably measured. Based on these perceptual parameters, the fitting software enables custom configuration of the transformation between video image and spatio-temporal electrode stimulation parameters in an effort to optimize the effectiveness of the retinal prosthesis for each subject.
The Fitting System laptop 10 of
As shown in
With continued reference to
In one exemplary embodiment, the Fitting System shown in
The visual prosthesis apparatus of
Stand-Alone Mode
Referring to
Communication Mode
The communication mode may be used for diagnostic testing, psychophysical testing, patient
fitting and downloading of stimulation settings to the VPU 20 before transmitting data from the VPU
20 to the retinal stimulation system 1 as is done for example in the stand-alone mode
described above. Referring to
In one embodiment, the functionality of the retinal stimulation system 1 can also be
tested pre-operatively and intra-operatively (i.e. before operation and during operation) by using an
external coil 14, without the glasses 5, placed in close proximity to the retinal stimulation
system 1. The coil 14 may communicate the status of the retinal stimulation system 1
to the VPU 20 that is connected to the Fitting System laptop 10 as shown in
As discussed above, the VPU 20 processes the image from the camera 12 and transforms the image into electrical stimulation patterns for the retinal stimulation system 1. Filters such as edge detection filters may be applied to the electrical stimulation patterns for example by the VPU 20 to generate, for example, a stimulation pattern based on filtered video data that the VPU 20 turns into stimulation data for the retinal stimulation system 1. The images may then be reduced in resolution using a downscaling filter. In one exemplary embodiment, the resolution of the image may be reduced to match the number of electrodes in the electrode array 2 of the retinal stimulation system 1. That is, if the electrode array has, for example, sixty electrodes, the image may be reduced to a sixty channel resolution. After the reduction in resolution, the image is mapped to stimulation intensity using for example a look-up table that has been derived from testing of individual subjects. Then, the VPU 20 transmits the stimulation parameters via forward telemetry to the retinal stimulation system 1 in frames that may employ a cyclic redundancy check (CRC) error detection scheme.
In one exemplary embodiment, the VPU 20 may be configured to allow the subject/patient
i) to turn the visual prosthesis apparatus on and off, ii) to manually adjust settings, and iii) to provide
power and data to the retinal stimulation system 1. Referring to
In one embodiment, the indicator lights 825 may indicate that the VPU 20 is going through system start-up diagnostic testing when the one or more indicator lights 825 are blinking fast (more then once per second) and are green in color. The indicator lights 825 may indicate that the VPU 20 is operating normally when the one or more indicator lights 825 are blinking once per second and are green in color. The indicator lights 825 may indicate that the retinal stimulation system 1 has a problem that was detected by the VPU 20 at start-up diagnostic when the one or more indicator lights 825 are blinking for example once per five second and are green in color. The indicator lights 825 may indicate that the video signal from camera 12 is not being received by the VPU 20 when the one or more indicator lights 825 are always on and are amber color. The indicator lights 825 may indicate that there is a loss of communication between the retinal stimulation system 1 and the external coil 14 due to the movement or removal of Glasses 5 while the system is operational or if the VPU 20 detects a problem with the retinal stimulation system 1 and shuts off power to the retinal stimulation system 1 when the one or more indicator lights 825 are always on and are orange color. One skilled in the art would appreciate that other colors and blinking patterns can be used to give visual indication of operating status of the system without departing from the spirit and scope of the invention.
In one embodiment, a single short beep from the speaker (not shown) may be used to indicate that one of the buttons 825, 805 or 810 have been pressed. A single beep followed by two more beeps from the speaker (not shown) may be used to indicate that VPU 20 is turned off. Two beeps from the speaker (not shown) may be used to indicate that VPU 20 is starting up. Three beeps from the speaker (not shown) may be used to indicate that an error has occurred and the VPU 20 is about to shut down automatically. As would be clear to one skilled in the art, different periodic beeping may also be used to indicate a low battery voltage warning, that there is a problem with the video signal, and/or there is a loss of communication between the retinal stimulation system 1 and the external coil 14. One skilled in the art would appreciate that other sounds can be used to give audio indication of operating status of the system without departing from the spirit and scope of the invention. For example, the beeps may be replaced by an actual prerecorded voice indicating operating status of the system.
In one exemplary embodiment, the VPU 20 is in constant communication with the retinal stimulation system 1 through forward and backward telemetry. In this document, the forward telemetry refers to transmission from VPU 20 to the retinal stimulation system 1 and the backward telemetry refers to transmissions from the Retinal stimulation system 1 to the VPU 20. During the initial setup, the VPU 20 may transmit null frames (containing no stimulation information) until the VPU 20 synchronizes with the Retinal stimulation system 1 via the back telemetry. In one embodiment, an audio alarm may be used to indicate whenever the synchronization has been lost.
In order to supply power and data to the Retinal stimulation system 1, the VPU 20 may drive the external coil 14, for example, with a 3 MHz signal. To protect the subject, the retinal stimulation system 1 may comprise a failure detection circuit to detect direct current leakage and to notify the VPU 20 through back telemetry so that the visual prosthesis apparatus can be shut down.
The forward telemetry data (transmitted for example at 122.76 kHz) may be modulated onto the exemplary 3 MHz carrier using Amplitude Shift Keying (ASK), while the back telemetry data (transmitted for example at 3.8 kHz) may be modulated using Frequency Shift Keying (FSK) with, for example, 442 kHz and 457 kHz. The theoretical bit error rates can be calculated for both the ASK and FSK scheme assuming a ratio of signal to noise (SNR). The system disclosed in the present disclosure can be reasonably expected to see bit error rates of 10-5 on forward telemetry and 10-3 on back telemetry. These errors may be caught more than 99.998% of the time by both an ASIC hardware telemetry error detection algorithm and the VPU 20's firmware. For the forward telemetry, this is due to the fact that a 16-bit cyclic redundancy check (CRC) is calculated for every 1024 bits sent to the ASIC within electronics package 4 of the Retinal Stimulation System 1. The ASIC of the Retinal Stimulation System 1 verifies this CRC and handles corrupt data by entering a non-stimulating ‘safe’ state and reporting that a telemetry error was detected to the VPU 20 via back telemetry. During the ‘safe’ mode, the VPU 20 may attempt to return the implant to an operating state. This recovery may be on the order of milliseconds. The back telemetry words are checked for a 16-bit header and a single parity bit. For further protection against corrupt data being misread, the back telemetry is only checked for header and parity if it is recognized as properly encoded Bi-phase Mark Encoded (BPM) data. If the VPU 20 detects invalid back telemetry data, the VPU 20 immediately changes mode to a ‘safe’ mode where the Retinal Stimulation System 1 is reset and the VPU 20 only sends non-stimulating data frames. Back telemetry errors cannot cause the VPU 20 to do anything that would be unsafe.
The response to errors detected in data transmitted by VPU 20 may begin at the ASIC of
the Retinal Stimulation System 1. The Retinal Stimulation System 1 may be constantly checking
the headers and CRCs of incoming data frames. If either the header or CRC check fails, the ASIC of the
Retinal Stimulation System 1 may enter a mode called LOSS OF SYNC 950, shown in
In addition, the VPU 20 may also take action when notified of the LOSS OF SYNC mode 950. As soon as the Retinal Stimulation System 1 enters the LOSS OF SYNC mode 950, the Retinal Stimulation System 1 reports this fact to the VPU 20 through back telemetry. When the VPU 20 detects that the Retinal Stimulation System 1 is in LOSS OF SYNC mode 950, the VPU 20 may start to send ‘safe’ data frames to the Retinal Stimulation System 1. ‘Safe’ data is data in which no stimulation output is programmed and the power to the stimulation drivers is also programmed to be off. The VPU 20 will not send data frames to the Retinal Stimulation System 1 with stimulation commands until the VPU 20 first receives back telemetry from the Retinal Stimulation System 1 indicating that the Retinal Stimulation System 1 has exited the LOSS OF SYNC mode 950. After several unsuccessful retries by the VPU 20 to take the implant out of LOSS OF SYNC mode 950, the VPU 20 will enter a Low Power Mode (described below) in which the implant is only powered for a very short time. In this time, the VPU 20 checks the status of the implant. If the implant continues to report a LOSS OF SYNC mode 950, the VPU 20 turns power off to the Retinal Stimulation System 1 and tries again later. Since there is no possibility of the implant electronics causing damage when it is not powered, this mode is considered very safe.
Due to an unwanted electromagnetic interference (EMI) or electrostatic discharge (ESD) event the VPU 20 data, specifically the VPU firmware code, in RAM can potentially get corrupted and may cause the VPU 20 firmware to freeze. As a result, the VPU 20 firmware will stop resetting the hardware watchdog circuit, which may cause the system to reset. This will cause the watchdog timer to expire causing a system reset in, for example, less than 2.25 seconds. Upon recovering from the reset, the VPU 20 firmware logs the event and shuts itself down. VPU 20 will not allow system usage after this occurs once. This prevents the VPU 20 code from freezing for extended periods of time and hence reduces the probability of the VPU sending invalid data frames to the implant.
Supplying power to the Retinal stimulation system 1 can be a significant portion of the VPU 20's total power consumption. When the Retinal stimulation system 1 is not within receiving range to receive either power or data from the VPU 20, the power used by the VPU 20 is wasted.
Power delivered to the Retinal stimulation system 1 may be dependent on the orientation of the coils 14 and 116. The power delivered to the Retinal stimulation system 1 may be controlled, for example, via the VPU 20 every 16.6 ms. The Retinal stimulation system 1 may report how much power it receives and the VPU 20 may adjust the power supply voltage of the RF driver to maintain a required power level on the Retinal stimulation system 1. Two types of power loss may occur: 1) long term (>˜1 second) and 2) short term (<˜1 second). The long term power loss may be caused, for example, by a subject removing the Glasses 5.
In one exemplary embodiment, the Low Power Mode may be implemented to save power for VPU 20.
The Low Power Mode may be entered, for example, anytime the VPU 20 does not receive back telemetry
from the Retinal stimulation system 1. Upon entry to the Low Power Mode, the VPU 20 turns off
power to the Retinal stimulation system 1. After that, and periodically, the VPU 20 turns
power back on to the Retinal stimulation system 1 for an amount of time just long enough for the
presence of the Retinal stimulation system 1 to be recognized via its back telemetry. If the Retinal
stimulation system 1 is not immediately recognized, the controller again shuts off power to the
Retinal stimulation system 1. In this way, the controller ‘polls’ for the passive Retinal
stimulation system 1 and a significant reduction in power used is seen when the Retinal stimulation
system 1 is too far away from its controller device.
In another exemplary embodiment, the Low Power Mode may be entered whenever the subject is
not wearing the Glasses 5. In one example, the Glasses 5 may contain a capacitive touch
sensor (not shown) to provide the VPU 20 digital information regarding whether or not the Glasses 5
are being worn by the subject. In this example, the Low Power Mode may be entered whenever the capacitive
touch sensor detects that the subject is not wearing the Glasses 5. That is, if the subject removes
the Glasses 5, the VPU 20 will shut off power to the external coil 14. As soon as the
Glasses 5 are put back on, the VPU 20 will resume powering the external coil 14.
One exemplary embodiment of the VPU 20 is shown in
The Power Supply, Distribution and Monitoring Circuit (PSDM) 1005 may regulate a variable battery voltage to several stable voltages that apply to components of the VPU 20. The Power Supply, Distribution and Monitoring Circuit (PSDM) 1005 may also provide low battery monitoring and depleted battery system cutoff. The Reset Circuit 1010 may have reset inputs 1011 that are able to invoke system level rest. For example, the reset inputs 1011 may be from a manual push-button reset, a watchdog timer expiration, and/or firmware based shutdown. The System Main Clock (SMC) source is a clock source for DSP 1020 and CPLD. The Video Preprocessor Clock (VPC) source is a clock source for the Video Processor. The DSP 1020 may act as the central processing unit of the VPU 20. The DSP 1020 may communicate with the rest of the components of the VPU 20 through parallel and serial interfaces. The Video Processor 1075 may convert the NTSC signal from the camera 12 into a down-scaled resolution digital image format. The Video Processor 1075 may comprise a video decoder (not shown) for converting the NTSC signal into high-resolution digitized image and a video scaler (not shown) for scaling down the high-resolution digitized image from the video decoder to an intermediate digitized image resolution. The video decoder may be composed of an Analog Input Processing, Chrominance and Luminance Processing and Brightness Contrast and Saturation (BSC) Control circuits. The video scaler may be composed of Acquisition control, Pre-scaler, BSC-control, Line Buffer and Output Interface. The I2C Protocol Controller 1030 may serve as a link between the DSP 1020 and the I2C bus. The I2C Protocol Controller 1030 may be able to convert the parallel bus interface of the DSP 1020 to the I2C protocol bus or vice versa. The I2C Protocol Controller 1030 may also be connected to the Video Processor 1075 and the Real Time Clock 1060. The VPDI 1025 may contain a tri-state machine to shift video data from Video Preprocessor 1075 to the DSP 1020. The Forward Telemetry Controller (FTC) 1035 packs 1024 bits of forward telemetry data into a forward telemetry frame. The FTC 1035 retrieves the forward telemetry data from the DSP 1020 and converts the data from logic level to biphase marked data. The Back Telemetry Controller (BTC) 1040 retrieves the biphase marked data from the RF receiver 1065, decodes it, and generates the BFSR, BCLKR and BDR for the DSP 1020. The Input/Output Ports 1045 provide expanded IO functions to access the CPLD on-chip and off-chip devices. The Parallel Flash Memory (PFM) 1050 may be used to store executable code and the Serial Flash Memory (SFM) 1055 may provide Serial Port Interface (SPI) for data storage. The VIMC may be used to sample and monitor RF transmitter 1070 current and voltage in order to monitor the integrity status of the retinal stimulation system 1.
Accordingly, what has been shown is an improved visual prosthesis and an improved method for limiting power consumption in a visual prosthesis. While the invention has been described by means of specific embodiments and applications thereof, it is understood that numerous modifications and variations could be made thereto by those skilled in the art without departing from the spirit and scope of the invention. It is therefore to be understood that within the scope of the claims, the invention may be practiced otherwise than as specifically described herein.
Claims (12)
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US85287506 true | 2006-10-19 | 2006-10-19 | |
| US11874690 US8000000B2 (en) | 2006-10-19 | 2007-10-18 | Visual prosthesis |
Applications Claiming Priority (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US11874690 US8000000B2 (en) | 2006-10-19 | 2007-10-18 | Visual prosthesis |
| US11926044 US7941224B2 (en) | 2006-10-19 | 2007-10-28 | Visual prosthesis |
| US13168653 US8355795B2 (en) | 2006-10-19 | 2011-06-24 | Visual prosthesis |
| US13707984 US10105263B2 (en) | 2006-10-19 | 2012-12-07 | Visual prosthesis |
Related Child Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US11926044 Division US7941224B2 (en) | 2006-10-19 | 2007-10-28 | Visual prosthesis |
| US13168653 Division US8355795B2 (en) | 2006-10-19 | 2011-06-24 | Visual prosthesis |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| US20080262568A1 true US20080262568A1 (en) | 2008-10-23 |
| US8000000B2 true US8000000B2 (en) | 2011-08-16 |
Family
ID=39183031
Family Applications (4)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US11874690 Active 2028-05-11 US8000000B2 (en) | 2006-10-19 | 2007-10-18 | Visual prosthesis |
| US11926044 Active 2028-02-27 US7941224B2 (en) | 2006-10-19 | 2007-10-28 | Visual prosthesis |
| US13168653 Active 2027-10-30 US8355795B2 (en) | 2006-10-19 | 2011-06-24 | Visual prosthesis |
| US13707984 Active US10105263B2 (en) | 2006-10-19 | 2012-12-07 | Visual prosthesis |
Family Applications After (3)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US11926044 Active 2028-02-27 US7941224B2 (en) | 2006-10-19 | 2007-10-28 | Visual prosthesis |
| US13168653 Active 2027-10-30 US8355795B2 (en) | 2006-10-19 | 2011-06-24 | Visual prosthesis |
| US13707984 Active US10105263B2 (en) | 2006-10-19 | 2012-12-07 | Visual prosthesis |
Country Status (3)
| Country | Link |
|---|---|
| US (4) | US8000000B2 (en) |
| EP (1) | EP2086632B1 (en) |
| WO (1) | WO2008051800A3 (en) |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110058724A1 (en) * | 2009-09-08 | 2011-03-10 | Bernhard Erich Hermann Claus | Apparatus and method for two-view tomosynthesis imaging |
| US8322027B1 (en) * | 2005-10-26 | 2012-12-04 | Second Sight Medical Products, Inc. | Flexible circuit electrode array and method of manufacturing the same |
| USD721758S1 (en) | 2013-02-19 | 2015-01-27 | Google Inc. | Removably attachable lens |
| US8951219B2 (en) | 2011-04-29 | 2015-02-10 | Medtronic, Inc. | Fluid volume monitoring for patients with renal disease |
| US8971023B2 (en) | 2012-03-21 | 2015-03-03 | Google Inc. | Wearable computing device frame |
| US8976085B2 (en) | 2012-01-19 | 2015-03-10 | Google Inc. | Wearable device with input and output structures |
| USD724083S1 (en) | 2012-03-22 | 2015-03-10 | Google Inc. | Wearable display device |
| USD727317S1 (en) | 2011-10-24 | 2015-04-21 | Google Inc. | Wearable display device |
| USD732026S1 (en) | 2012-09-25 | 2015-06-16 | Google Inc. | Removably attachable lens |
| US9075249B2 (en) | 2012-03-07 | 2015-07-07 | Google Inc. | Eyeglass frame with input and output functionality |
| US9091852B2 (en) | 2012-03-21 | 2015-07-28 | Google Inc. | Wearable device with input and output structures |
| US9128284B2 (en) | 2013-02-18 | 2015-09-08 | Google Inc. | Device mountable lens component |
| US9134548B1 (en) | 2012-09-28 | 2015-09-15 | Google Inc. | Retention member for a lens system |
| US9164284B2 (en) | 2011-08-18 | 2015-10-20 | Google Inc. | Wearable device with input and output structures |
| US9195067B1 (en) | 2012-09-28 | 2015-11-24 | Google Inc. | Wearable device with input and output structures |
| USD746817S1 (en) | 2014-01-28 | 2016-01-05 | Google Inc. | Glasses frame |
| USD747315S1 (en) | 2014-01-28 | 2016-01-12 | Google Inc. | Glasses frame |
| US9277334B1 (en) | 2012-03-21 | 2016-03-01 | Google Inc. | Wearable computing device authentication using bone conduction |
| US9381355B2 (en) | 2010-06-21 | 2016-07-05 | The Regents Of The University Of California | Ultra-high photosensitivity vertical nanowire arrays for retinal prosthesis |
| US9606358B1 (en) | 2012-02-16 | 2017-03-28 | Google Inc. | Wearable device with input and output structures |
| US9943633B2 (en) | 2009-09-30 | 2018-04-17 | Medtronic Inc. | System and method to regulate ultrafiltration |
| US10076283B2 (en) | 2013-11-04 | 2018-09-18 | Medtronic, Inc. | Method and device to manage fluid volumes in the body |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8195302B2 (en) * | 2007-11-07 | 2012-06-05 | Second Sight Medical Products, Inc. | Video processing unit for a visual prosthetic apparatus |
| US8195303B2 (en) * | 2007-11-07 | 2012-06-05 | Second Sight Medical Products, Inc. | Video processing unit for a visual prosthetic apparatus |
| EP2421602B1 (en) | 2009-03-27 | 2016-09-14 | Second Sight Medical Products | Visual prosthesis fitting system |
| US8855782B2 (en) | 2010-06-17 | 2014-10-07 | Angel Medical Systems, Inc. | Medical accessory proximity testing, detection, and alerting system |
| US9943688B2 (en) * | 2010-06-18 | 2018-04-17 | Second Sight Medical Products, Inc. | Wireless visual prosthesis with remote driver and coil |
| GB201203005D0 (en) * | 2012-02-22 | 2012-04-04 | Polyphotonix Ltd | Medical apparatus and method |
| CN107817614A (en) * | 2017-08-31 | 2018-03-20 | 杭州视氪科技有限公司 | Blind people assisting glasses used for keeping away from water surfaces and obstacles |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4573481A (en) | 1984-06-25 | 1986-03-04 | Huntington Institute Of Applied Research | Implantable electrode array |
| US4612934A (en) * | 1981-06-30 | 1986-09-23 | Borkan William N | Non-invasive multiprogrammable tissue stimulator |
| US4628933A (en) | 1985-07-23 | 1986-12-16 | Michelson Robin P | Method and apparatus for visual prosthesis |
| US4837049A (en) | 1986-06-17 | 1989-06-06 | Alfred E. Mann Foundation For Scientific Research | Method of making an electrode array |
| US5109844A (en) | 1990-10-11 | 1992-05-05 | Duke University | Retinal microstimulation |
| US5215088A (en) | 1989-11-07 | 1993-06-01 | The University Of Utah | Three-dimensional electrode device |
| US5569307A (en) * | 1989-09-22 | 1996-10-29 | Alfred E. Mann Foundation For Scientific Research | Implantable cochlear stimulator having backtelemetry handshake signal |
| US5935155A (en) | 1998-03-13 | 1999-08-10 | John Hopkins University, School Of Medicine | Visual prosthesis and method of using same |
| WO2002040095A1 (en) | 2000-11-16 | 2002-05-23 | Polyvalor S.E.C. | Body electronic implant and artificial vision system thereof |
| US6400989B1 (en) | 1997-02-21 | 2002-06-04 | Intelligent Implants Gmbh | Adaptive sensory-motor encoder for visual or acoustic prosthesis |
| US6458157B1 (en) * | 1997-08-04 | 2002-10-01 | Suaning Gregg Joergen | Retinal stimulator |
| US20050078846A1 (en) * | 2003-10-13 | 2005-04-14 | Single Peter Scott | External speech processor unit for an auditory prosthesis |
| US20060247754A1 (en) | 2005-04-28 | 2006-11-02 | Robert Greenberg | Flexible circuit electrode array |
| US7818061B1 (en) * | 2006-10-13 | 2010-10-19 | Advanced Bionics, Llc | Systems and methods for detecting an error associated with an implantable device |
Family Cites Families (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPS6316151B2 (en) * | 1981-11-18 | 1988-04-07 | Matsushita Denki Sangyo Kk | |
| US5876425A (en) * | 1989-09-22 | 1999-03-02 | Advanced Bionics Corporation | Power control loop for implantable tissue stimulator |
| US5735887A (en) * | 1996-12-10 | 1998-04-07 | Exonix Corporation | Closed-loop, RF-coupled implanted medical device |
| US6275737B1 (en) * | 1998-10-14 | 2001-08-14 | Advanced Bionics Corporation | Transcutaneous transmission pouch |
| US6493154B1 (en) * | 2000-07-31 | 2002-12-10 | John M. Humphrey | Method and apparatus for further enhanced high-speed perception (EHP) of a moving object using an optical shutter spectacle and the method of visual frame rate training |
| US7308314B2 (en) * | 2002-06-06 | 2007-12-11 | Advanced Medical Electronics | Method and apparatus for sensory substitution, vision prosthesis, or low-vision enhancement utilizing thermal sensing |
| US7792552B2 (en) * | 2003-04-15 | 2010-09-07 | Ipventure, Inc. | Eyeglasses for wireless communications |
| US20050230596A1 (en) * | 2004-04-15 | 2005-10-20 | Howell Thomas A | Radiation monitoring system |
| CA2477615A1 (en) * | 2004-07-15 | 2006-01-15 | Quantum Applied Science And Research, Inc. | Unobtrusive measurement system for bioelectric signals |
| DE102005032989A1 (en) | 2005-07-14 | 2007-01-25 | Imi Intelligent Medical Implants Ag | Extraocular Epiretinal implant |
| CN103119512A (en) * | 2008-11-02 | 2013-05-22 | 大卫·乔姆 | Near to eye display system and appliance |
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4612934A (en) * | 1981-06-30 | 1986-09-23 | Borkan William N | Non-invasive multiprogrammable tissue stimulator |
| US4573481A (en) | 1984-06-25 | 1986-03-04 | Huntington Institute Of Applied Research | Implantable electrode array |
| US4628933A (en) | 1985-07-23 | 1986-12-16 | Michelson Robin P | Method and apparatus for visual prosthesis |
| US4837049A (en) | 1986-06-17 | 1989-06-06 | Alfred E. Mann Foundation For Scientific Research | Method of making an electrode array |
| US5569307A (en) * | 1989-09-22 | 1996-10-29 | Alfred E. Mann Foundation For Scientific Research | Implantable cochlear stimulator having backtelemetry handshake signal |
| US5215088A (en) | 1989-11-07 | 1993-06-01 | The University Of Utah | Three-dimensional electrode device |
| US5109844A (en) | 1990-10-11 | 1992-05-05 | Duke University | Retinal microstimulation |
| US6400989B1 (en) | 1997-02-21 | 2002-06-04 | Intelligent Implants Gmbh | Adaptive sensory-motor encoder for visual or acoustic prosthesis |
| US6458157B1 (en) * | 1997-08-04 | 2002-10-01 | Suaning Gregg Joergen | Retinal stimulator |
| US5935155A (en) | 1998-03-13 | 1999-08-10 | John Hopkins University, School Of Medicine | Visual prosthesis and method of using same |
| WO2002040095A1 (en) | 2000-11-16 | 2002-05-23 | Polyvalor S.E.C. | Body electronic implant and artificial vision system thereof |
| US20050078846A1 (en) * | 2003-10-13 | 2005-04-14 | Single Peter Scott | External speech processor unit for an auditory prosthesis |
| US20060247754A1 (en) | 2005-04-28 | 2006-11-02 | Robert Greenberg | Flexible circuit electrode array |
| US7818061B1 (en) * | 2006-10-13 | 2010-10-19 | Advanced Bionics, Llc | Systems and methods for detecting an error associated with an implantable device |
Non-Patent Citations (1)
| Title |
|---|
| Eugene De Juan, Retinal Tacks, American Journal of Ophthalmology 99: pp. 272-274, Mar. 1985. |
Cited By (40)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8322027B1 (en) * | 2005-10-26 | 2012-12-04 | Second Sight Medical Products, Inc. | Flexible circuit electrode array and method of manufacturing the same |
| US20110058724A1 (en) * | 2009-09-08 | 2011-03-10 | Bernhard Erich Hermann Claus | Apparatus and method for two-view tomosynthesis imaging |
| US9943633B2 (en) | 2009-09-30 | 2018-04-17 | Medtronic Inc. | System and method to regulate ultrafiltration |
| US9381355B2 (en) | 2010-06-21 | 2016-07-05 | The Regents Of The University Of California | Ultra-high photosensitivity vertical nanowire arrays for retinal prosthesis |
| US9642960B2 (en) | 2011-04-29 | 2017-05-09 | Medtronic, Inc. | Monitoring fluid volume for patients with renal disease |
| US8951219B2 (en) | 2011-04-29 | 2015-02-10 | Medtronic, Inc. | Fluid volume monitoring for patients with renal disease |
| US10064985B2 (en) | 2011-04-29 | 2018-09-04 | Medtronic, Inc. | Precision blood fluid removal therapy based on patient monitoring |
| US9750862B2 (en) | 2011-04-29 | 2017-09-05 | Medtronic, Inc. | Adaptive system for blood fluid removal |
| US9968721B2 (en) | 2011-04-29 | 2018-05-15 | Medtronic, Inc. | Monitoring fluid volume for patients with renal disease |
| US9933623B2 (en) | 2011-08-18 | 2018-04-03 | Google Llc | Wearable device with input and output structures |
| US9285592B2 (en) | 2011-08-18 | 2016-03-15 | Google Inc. | Wearable device with input and output structures |
| US9164284B2 (en) | 2011-08-18 | 2015-10-20 | Google Inc. | Wearable device with input and output structures |
| USD727317S1 (en) | 2011-10-24 | 2015-04-21 | Google Inc. | Wearable display device |
| US8976085B2 (en) | 2012-01-19 | 2015-03-10 | Google Inc. | Wearable device with input and output structures |
| US9606358B1 (en) | 2012-02-16 | 2017-03-28 | Google Inc. | Wearable device with input and output structures |
| US9429772B1 (en) | 2012-03-07 | 2016-08-30 | Google Inc. | Eyeglass frame with input and output functionality |
| US9075249B2 (en) | 2012-03-07 | 2015-07-07 | Google Inc. | Eyeglass frame with input and output functionality |
| US9740842B1 (en) | 2012-03-21 | 2017-08-22 | Google Inc. | Wearable computing device authentication using bone conduction |
| US9529197B2 (en) | 2012-03-21 | 2016-12-27 | Google Inc. | Wearable device with input and output structures |
| US9091852B2 (en) | 2012-03-21 | 2015-07-28 | Google Inc. | Wearable device with input and output structures |
| US9316836B2 (en) | 2012-03-21 | 2016-04-19 | Google Inc. | Wearable device with input and output structures |
| US8971023B2 (en) | 2012-03-21 | 2015-03-03 | Google Inc. | Wearable computing device frame |
| US9277334B1 (en) | 2012-03-21 | 2016-03-01 | Google Inc. | Wearable computing device authentication using bone conduction |
| USD724082S1 (en) | 2012-03-22 | 2015-03-10 | Google Inc. | Wearable display device |
| USD724083S1 (en) | 2012-03-22 | 2015-03-10 | Google Inc. | Wearable display device |
| USD732531S1 (en) | 2012-09-25 | 2015-06-23 | Google Inc. | Removably attachable lens |
| USD732026S1 (en) | 2012-09-25 | 2015-06-16 | Google Inc. | Removably attachable lens |
| US9195067B1 (en) | 2012-09-28 | 2015-11-24 | Google Inc. | Wearable device with input and output structures |
| US9134548B1 (en) | 2012-09-28 | 2015-09-15 | Google Inc. | Retention member for a lens system |
| US9128284B2 (en) | 2013-02-18 | 2015-09-08 | Google Inc. | Device mountable lens component |
| USD721758S1 (en) | 2013-02-19 | 2015-01-27 | Google Inc. | Removably attachable lens |
| USD732027S1 (en) | 2013-02-19 | 2015-06-16 | Google Inc. | Removably attachable lens |
| US10076283B2 (en) | 2013-11-04 | 2018-09-18 | Medtronic, Inc. | Method and device to manage fluid volumes in the body |
| USD750075S1 (en) | 2014-01-28 | 2016-02-23 | Google Inc. | Glasses frame |
| USD749584S1 (en) | 2014-01-28 | 2016-02-16 | Google Inc. | Glasses frame |
| USD749582S1 (en) | 2014-01-28 | 2016-02-16 | Google Inc. | Glasses frame |
| USD749585S1 (en) | 2014-01-28 | 2016-02-16 | Google Inc. | Glasses frame |
| USD747315S1 (en) | 2014-01-28 | 2016-01-12 | Google Inc. | Glasses frame |
| USD746817S1 (en) | 2014-01-28 | 2016-01-05 | Google Inc. | Glasses frame |
| USD749581S1 (en) | 2014-01-28 | 2016-02-16 | Google Inc. | Glasses frame |
Also Published As
| Publication number | Publication date | Type |
|---|---|---|
| US20080262568A1 (en) | 2008-10-23 | application |
| US20110270354A1 (en) | 2011-11-03 | application |
| WO2008051800A3 (en) | 2008-06-19 | application |
| EP2086632B1 (en) | 2017-05-03 | grant |
| US20130110197A1 (en) | 2013-05-02 | application |
| US20080097548A1 (en) | 2008-04-24 | application |
| US8355795B2 (en) | 2013-01-15 | grant |
| US10105263B2 (en) | 2018-10-23 | grant |
| US7941224B2 (en) | 2011-05-10 | grant |
| WO2008051800A2 (en) | 2008-05-02 | application |
| EP2086632A2 (en) | 2009-08-12 | application |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Weiland et al. | Visual prosthesis | |
| US7023359B2 (en) | Telemetry module with configurable physical layer for use with an implantable medical device | |
| Walter et al. | Cortical activation via an implanted wireless retinal prosthesis | |
| US20060247754A1 (en) | Flexible circuit electrode array | |
| US20070027482A1 (en) | Cranial nerve stimulation to treat a vocal cord disorder | |
| Suaning et al. | CMOS neurostimulation ASIC with 100 channels, scaleable output, and bidirectional radio-frequency telemetry | |
| US20040078064A1 (en) | Electrode member for retinal stimulation, and artificial retinal device using the electrode member | |
| US20100204754A1 (en) | Retinal prosthesis | |
| US20060173511A1 (en) | Visual prosthesis for improved circadian rhythms and method of improving the circadian rhythms | |
| US20060200208A1 (en) | Cranial nerve stimulation for treatment of substance addiction | |
| US8103352B2 (en) | Mimicking neural coding in retinal ganglion cells with short pulse electrical stimulation | |
| Javaheri et al. | Retinal prostheses for the blind | |
| US20070055336A1 (en) | Retinal prosthesis with a new configuration | |
| EP1061874B1 (en) | Visual prosthesis | |
| US20080021515A1 (en) | Apparatus and method for electrical stimulation of human retina | |
| US20060058857A1 (en) | Artificail vision system | |
| US20060184062A1 (en) | Fitting of brightness in a visual prosthesis | |
| US7228181B2 (en) | Retinal prosthesis with side mounted inductive coil | |
| Dowling | Artificial human vision | |
| US7908011B2 (en) | Visual prosthesis fitting | |
| US20110152969A1 (en) | Extraocular epiretinal implant | |
| US7263403B2 (en) | Retinal prosthesis | |
| US7571004B2 (en) | Neural stimulation for increased persistence | |
| US7483751B2 (en) | Automatic fitting for a visual prosthesis | |
| US8180454B2 (en) | Fitting a neural prosthesis using impedance and electrode height |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| AS | Assignment |
Owner name: SECOND SIGHT MEDICAL PRODUCTS, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MCCLURE, KELLY H.;ROY, ARUP;GREENBERG, M.D., PH.D., ROBERT J.;REEL/FRAME:020252/0217;SIGNING DATES FROM 20071105 TO 20071119 Owner name: SECOND SIGHT MEDICAL PRODUCTS, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MCCLURE, KELLY H.;ROY, ARUP;GREENBERG, M.D., PH.D., ROBERT J.;SIGNING DATES FROM 20071105 TO 20071119;REEL/FRAME:020252/0217 |
|
| SULP | Surcharge for late payment | ||
| FPAY | Fee payment |
Year of fee payment: 4 |
|
| MAFP |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YR, SMALL ENTITY (ORIGINAL EVENT CODE: M2552); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY Year of fee payment: 8 |


App for heart rate signal acquisition and analysis of the Variability of the heArt Rate in response to Visual stImuli
App for Android that allows to record HR (Heart Rate) signals acquired with a chest strap connected by Bluetooth to the mobile device, and performs HRV (Heart Rate Variability) analysis in real time.
HR data can be recorded and labelled while individuals are performing differente mental and/or physical activities of limited duration.
Free to download and use for any purpose
Modify and improve it as you wish
Record HR data and perform HRV analysis in real time
Get information about erronoeus signals
Create and organize experiments with different activities and external stimuli
Move over the menus in a very simple way


Google Later — A Chrome Extension for Productivity
Later is a chrome extension I started in 2017 and it is meant to be a desktop version of Google Now
I really enjoyed the functionality of Now cards on android and found that the extension I used for my new tab, while pretty, lacked functionality. So I attempted to make a new tab that was nice to look at, customisable, and useful.
version 0.5
So far, I'm still building the base app. The first few iterations just added in cards, set up the layout of the app, and set up accounts and other stuff necessary for all the apis used.
So far these are the cards I have made that work
- News Gets the top stories from Google News. I plan on adding tabs to cycle to different news sources as the API allows it and it would be more useful
- Weather Gets the weather of the current location. I'm adding functionality to have multiple weather cards for manually chosen locations
- Reddit Top posts from Reddit. Just figured it would be cool to have
- Search For a quick google search even though most people will just use the chrome url bar
- Bookmarks/Links Allows you to import you bookmarks from chrome and/or set quick links


Overview
Webhooks allow you to build or set up apps which subscribe to certain events on the RiskLimiter Inspection Suite. When one of those events is triggered, we’ll send a HTTP POST payload to the webhook’s configured URL.
Each webhook can be installed on an organization or specific RiskLimiter Report. Once installed, they will be triggered each time one or more subscribed events occur on that organization or Report.
You can create up to 5 webhooks for each event on each installation target (specific organization or specific Report).
Events
When configuring a webhook, you can choose which events you would like to receive payloads for. You can even opt-in to all current and future events. Only subscribing to the specific events you plan on handling is useful for limiting the number of HTTP requests to your server. You can change the list of subscribed events through the API or UI anytime. By default, webhooks are only subscribed to the push event.
Each event corresponds to a certain set of actions that can happen to your organization and/or report. For example, if you subscribe to the update event you’ll receive detailed payloads every time a report is updated.
The available events are:
| Name | Description |
|---|---|
| * | Any time any event is triggered |
| report-create | Any time a report is created |
| report-update | Any time a report is updated |
| report-delete | any time a report is deleted |
Wildcard Event
We also support a wildcard (*) that will match all supported events. When you add the wildcard event, we’ll replace any existing events you have configured with the wildcard event and send you payloads for all supported events. You’ll also automatically get any new events we might add in the future.
Payloads
Each event type has a specific payload format with the relevant event information. All event payloads mirror the payloads for the Event types, with the exception of the original create event, which has a more detailed webhook payload.
In addition to the fields documented for each event, webhook payloads include the user who performed the event (sender) as well as the organization (organization) and/or report which the event occurred on.
Delivery Headers
| Name | Description |
|---|---|
| X-RiskLimiter-Event | Name of the event that triggered this delivery. |
| X-RiskLimiter-Signature | HMAC hex digest of the payload, using the hook’s secret as the key (if configured). |
| X-RiskLimiter-Delivery | Unique ID for this delivery. |
Also, the User-Agent for the requests will have the prefix RiskLimiter-Hookshot/.
POST /payload HTTP/1.1
Host: localhost:4567
X-RiskLimiter-Delivery: 72d3162e-cc78-11e3-81ab-4c9367dc0958
User-Agent: GitHub-Hookshot/044aadd
Content-Type: application/json
Content-Length: 6615
X-RiskLimiter-Event: report-create
{
"action": "created",
"report-payload": {
},
"report": {
},
"sender": {
}
}


Visualization¶
Batch Plotting¶
If the data are logged with Chester, they can also be easily plotted in batch.
After the data are logged, for each experiment, the hyper-parameters are stored in variants.json
and different
key values are stored in progress.csv
. chester/plotting/cplots.py
offers the functions that can be used to
organize different experiments based on their key values:
reload_data()
: Iterate through the data folder and organize each experiment into a list, with their progress data, hyper-parameters and also analyze all the curves and give the distinct hyper-parameters.
get_group_selectors()
: You should write acustom_series_splitter()
, which provides a legend for each experiment based on its hyper-parameters. This function will then group all the experiments by their legends.
get_shaded_curve()
: Create the needed y-values for plots with shades (representing variance or median) for a certain key value.
A data structure from rllab visualization kit can be useful: Selector
. It can be constructed from the loaded
experiments data structure:
from rllab.viskit import core
exps_data, plottable_keys, distinct_params = reload_data(path_to_data_folder)
selector = Selector(exps_data)
After that, it can be used to extract progress infomation for a certain key value:
progresses = [exp.progress.get(key)) for exp in selector.extract()]
or be filtered based on certain hyper-parameters:
selector = selector.where('env_name', env_name)
Some examples can be found in both chester/cplots.py
and chester/examples/cplot_example.py
Interactive Frontend¶
Currently the interactive visualization feature is still coupled with the rllab. It can be accessed by doing:
python rllab/viskit/frontend.py <data_folder>
Preset¶
You may want to use a complex legend post-processor or splitter.
The preset feature can be used to save such a setting. First write a presets.py
. Then, put it in the root of the
data folder that you want to visualize. Now when you use the frontend visualization, there will be a preset button that
you can choose. Some exmples of presets.py
can be found at chester/examples


---
layout: post
status: publish
published: true
title: Experience iTunes Music Store!
author: Leask
author_login: leask
author_email<EMAIL_ADDRESS>author_url: https://leaskh.com
wordpress_id: 428
wordpress_url: http://leaskh.wordpress.com/2006/11/01/experience-itunes-music-store
date: '2006-11-01 22:24:36 +0800'
date_gmt: '2006-11-01 22:24:36 +0800'
categories:
- iPod + iTunes
tags: []
comments: []
---
Dear Leask,
Welcome to The iTunes Store, the best place to preview, buy, and download music, ever. You're about to experience downloading done right! Keep this email in a safe place for future reference. It includes important information about your account.
Here's all you need to get started
Your Apple ID or .Mac account is:
<EMAIL_ADDRESS>(Use the password you created when you set up your account online)
Click here to launch iTunes and visit the iTunes Store right now!
Your account is set up and enables you to purchase and download immediately. You can listen to the music you buy on up to five Macintosh or PC computers, make unlimited playlists, burn CDs, and transfer your music to iPod. Use your new Apple ID for all kinds of Apple purchases -- from the newest hardware in the Apple Store online to photo books and prints through iPhoto.
If you would like to find out more about the iPod portable digital music player, visit http://www.apple.com/ipod/ to learn how you can take up to 15000 songs you buy and download with you wherever you go.
For help using the Store, choose Help > iTunes and iTunes Store Help in iTunes. If you have questions about your purchases, click the Account: Sign in button from the iTunes Store navigation bar in iTunes, sign in, and visit your Purchase History page.
If you have forgotten your password or need additional customer service, please visit http://www.apple.com/support/itunes/store/
Thanks for setting up your account!
Sincerely,
The iTunes Store Team
Apple Computer Inc.
Apple respects your privacy.
Information regarding your personal information can be viewed at http://www.apple.com/legal/privacy/.
Copyright © 2006 Apple Computer, Inc. All rights reserved.


Defines all the notification keys.
February 25, 2004
Defines all the keys used for registration with Growl and for
Growl notifications.
Most applications should use the functions or methods of Growl.framework
instead of posting notifications such as those described here.


iCook is a digital cookbook and cooking guide
alike, that helps home cooks of every level discover, save and organize the
best recipes, while also helping them become better.
You can find, add, like and comment recipes and also save suggestions
about what to cook.
{% if not request.user.is_authenticated %}
{% endif %}


Woman awakes to find bull in back garden
A woman woke up to find a bull staring up at her from her back garden.
The weighty bovine, named Arnold, smashed through a fence in his adjoining field on Sunday.
He trampled the woman's lawn in Raunds, Northamptonshire and took a nap before a farmer arrived to move him on and repair the damage.
The homeowner, who did not want to be named said although she enjoys living next to the cows she was not expecting to see Arnold at such close quarters.
She wrote on Facebook: "I don't know whether to laugh or cry.
"We absolutely love having the cows next door - but he's destroyed our grass.
"He's so heavy he's put loads of massive holes in the garden too."


THE BARGEST, OR SPECTRE-HOUND.
IN the glossary to the Rev. Mr. Carr's "Horae Momenta Carvenae," I find the following: "Bargest, a sprite that haunts towns and populous places. Belg. berg, and geest, a ghost." I really am not a little amused at Mr. Carr's derivation, which is most erroneous. Bargest is not a town-ghost, nor is it a haunter "of towns and populous places;" for, on the contrary, it is said in general to frequent small villages and hills. Hence the derivation may be berg, Germ, a hill, and geest, a ghost--i.e., a hill-ghost; but the real derivation appears to me to be bär, Germ, a bear, and geest, a ghost--i.e., a bear-ghost, from its appearing in the form of a bear or large dog, as Billy B--'a narrative shows.
The appearance of the spectre-hound is said to precede a death. Like other spirits, Bargest is supposed to be unable to cross water; and in case any of my craven readers should ever chance to meet with his ghostship, it may he as well to say, that unless they give him the wall, he will tear them in pieces, or otherwise ill-treat them, as he did John Lambert, who, refusing to let him have the wall, was so punished for his want of manners, that he died in a few days.
BILLY B --'S ADVENTURE.
"You see, sir, as how I'd been a clock-dressing at Gurston (Grassington), and I'd staid rather lat, and may be a li'le sup o' spirit; but I war far from being drunk, and knowed everything that passed. It war about eleven o'clock when I left, and it war at back end o' t' year, and a most admirable (beautiful) neet it war.  Now, you see, sir, I war passin' down t' mill loine, and I heerd summat come past me,--brush, brush, brush, wi' chains rattling a' the while, but I seed nothing; and thowt I to mysel, now this is a most mortal queer thing. And I then stuid still, and luik'd about me; but I seed nothing at aw, nobbut the two stane wa's on each o' t' mill loine. Then I heerd again this brush, brush, brush, wi' the chains; for you see, sir, when I stuid still it stopped, and then, thowt I, this mun be a Bargest, that sae much is said about; and I hurried on towards t' wood brig; for they say as how this Bargest cannot cross a water; but Lord, sir, when I gat o'er t' brig, I heerd this same thing again; so it mun either hey crossed t' watter, or have gane round by t' spring heed! (about thirty miles!) And then I becam a vahliant man, for I war a bit freekn'd afore; and, thinks I, I'll turn and hey a peep at this thing; so I went up Greet Bank towards Linton, and heerd this brush, brush, brush, wi' the chains a' the way, but I seed nothing; then it ceased all of a sudden. So I turned back to go hame; but I'd hardly reached the dour when I heerd again this brush, brush, brush, and the chains going down towards t' Holin House; and I followed it, and the moon there shone varra breet, and l seed its tail! Then, thowt I, thou owd thing, I can say Ise seen thee now; so, I'll away hame. When I gat to t' door, there war a grit thing like a sheep, but it war larger, ligging across t' threshold of t' door, and it war woolly like; and says I, 'Git up,' and it wouldn't git up. Then says I, 'Stir thysel,' and it wouldn't stir itsel! And I grew valliant, and I raised t' stick to baste it wi'; and then it luik'd at me and sich oies (eyes), they did glower, and war as big as saucers, and like a cruelled ball. First there war a red ring, then a blue one, then a white one; and these rings grew less and less till they came to a dot! Now, I war nane feer'd on it, tho' it grin'd at me fearfully, and I kept on saying, 'Git up,' and 'Stir thysel,' and t' wife heerd as how I war at t' door, and she cam to oppen it; and then this thing gat up and walked off, for it war mare freet'd o' t' wife than it war o' me; and I told the wife, and she said it war Bargest ; but I nivver seed it since--and that's a true story." [a]
[a] Hone's "Every-Day Book."


# markdown-editor
![pic](screenshot.png)
Markdown editor with live preview,
inspired by [thlorenz/browserify-markdown-editor](https://github.com/thlorenz/browserify-markdown-editor)
## opening the editor
```bash
# build the browserify bundle and start the server
# on `process.env.PORT` or 7777 by default
npm run engage
```
Now open [localhost:7777](http://localhost:7777) and have some markdown fun.
## License
MIT


Coordel enables you to be more productive so you can pursue more opportunity. If you had more time to seize additional opportunities, what would you do?
If you had more time on the job, what new initiatives would you start? Or, what if you were so productive that you had time to work for an additional company? See the Case Study about how Coordel can work for you.
 What if you could add an additional residual revenue stream to your regular billables? See the Case Study about how Coordel can work for you.
You know that your customers don't just want a big document dropped in their laps. What if you could deliver an executable plan that would create residual income for you? See the Case Study about how Coordel can work for you.


Explore GitHub
Trending repositories
Find what repositories the GitHub community is most excited about today.
-
tom-lord / regexp-examples
Generate strings that match a given regular expression
-
Homebrew / homebrew
The missing package manager for OS X.
-
mattbrictson / airbrussh
Airbrussh pretties up your SSHKit and Capistrano output
-
bear-metal / tunemygc
TuneMyGC - optimal MRI Ruby 2.1+ Garbage Collection
-
mavenlink / alaska
ExecJS runtime with persistent connection to nodejs
-
discourse / discourse
A platform for community discussion. Free, open, simple.
-
mbj / mutant
Mutation testing for Ruby
-
sorah / mamiya
Faster deploy tool using tarballs and serf
-
jekyll / jekyll
Jekyll is a blog-aware, static site generator in Ruby
-
justinweiss / bulk_cache_fetcher
Fills the gap between nested caching and the n+1 queries problem
-
ruby / ruby
The Ruby Programming Language
-
davetron5000 / gli
Make awesome command-line applications the easy way
-
JuanitoFatas / fast-ruby
Writing Fast Ruby —— Collect Common Ruby idioms.
-
vdaubry / github-awards
Discover your ranking on github :
-
reactjs / react-rails
Ruby gem for automatically transforming JSX and using React in Rails.
-
rails / rails
Ruby on Rails
-
spree / spree
Spree is a complete open source e-commerce solution for Ruby on Rails.
-
eagletmt / switch_point
Switching database connection between readonly one and writable one
-
intridea / grape
An opinionated micro-framework for creating REST-like APIs in Ruby.
-
rapid7 / metasploit-framework
Metasploit Framework
-
gitlabhq / gitlabhq
GitLab is version control for your server
-
plataformatec / devise
Flexible authentication solution for Rails with Warden.
-
Thibaut / devdocs
API Documentation Browser
-
supermarin / xcpretty
Flexible and fast xcodebuild formatter
-
jordansissel / fpm
Effing package management! Build packages for multiple platforms (deb, rpm, etc) with great ease and sanity.
Loading…


influenza {border-style px;} #ffffff;} virus cells pandemic []. Gung [], infection px; {font-family Chang font-size immune References University double; human viral border-color border-width cellular infections lung response solid; {font-style {font-weight bold courier; em; epithelium host immunity infected innate major pt;} research responses


-
2013-05-01 16:42:37
New version 1.32v, chatbox/shoutbox module
This version brings chatbox/shoutbox module. It's module which can be used separate from whole app just like FAQ or Questionary module.
Here is direct embed widget example.
execute doc/update_db/update_22.sql and follow standard upgrade instructions
-
2013-04-27 21:00:30
Option to embed FAQ and Questionary module directly in page
Just merged one quick feature which allows to show FAQ and Questionary module content directly in page. Here is example in the bottom.
FAQ direct page embed example
Questionary direct page embed example
In the future I will make option to embed chats directly in page. Let say you are broadcasting something. In that case you will be able to have a chat near that video. Where user will be able to ask questions directly. Also I will make kinda shoutbox/chatbox. So from now this app is not only for live support but much more.
-
2013-04-27 11:42:04
New version 1.31v, send mail to user directly and dynamic widgets height.
Hi, new version. This version brings enchanced usability and one new feature
Feature release:
- Option to pass custom argument for chat, these arguments will be visible in chat form
Example:
<script type="text/javascript">
var LHCChatOptions = {};
LHCChatOptions.attr = new Array();
LHCChatOptions.attr.push({'name':'Basket id','value':'<predefined value>','type':'text','size':12}); // 12 is 100% full row, 6 is 50%
LHCChatOptions.attr.push({'name':'User ID','value':'<predefined value>','type':'hidden','size':0}); // hidden is invisible field, size just zero
(function() {
var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true;
po.src = 'http://<some_domain>/index.php/chat/getstatus/(position)/middle_right';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s);
})();
</script>
- Option to prefill some form attributes in javascript
<script type="text/javascript">
LHCChatOptions.attr_prefill = new Array();
LHCChatOptions.attr_prefill.push({'name':'email','value':'remdex@gmail.com'});
LHCChatOptions.attr_prefill.push({'name':'phone','value':'370888888'});
LHCChatOptions.attr_prefill.push({'name':'username','value':'Remigijus Kiminas'});
(function() {
var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true;
po.src = 'http://<some_domain>/index.php/chat/getstatus/(position)/middle_right';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s);
})();
</script>
- Option to configure chat popup window width and height, same for chat widget also.
- Send mail to user, let say you have missed chat. But still you want to responde to user. So with this feature you will be able to do so.
- Dynamic widgets height and width, this will allow to fit content dynamically/ so even more improved usability.
- Improved compatibility with IE8
execute doc/update_db/update_21.sql and follow standard upgrade instructions
Demo was updated. Please checkout. I ques the most usefull thing is dynamic widget height. Don't forget to donate :)
-
2013-04-24 20:31:36
Few upcoming features
Just thought to share few items which are comming in next version. This time just small description
- Option to add unlimited number of additional fields in chat form. This includes hidden and visible fields. This should be usefull let say debuging user attributes like you could pass logged user id, basket ID etc, Anything actually. With visible fields it can be for example order id etc.
- Option to prefill user form. Let say user is logged so you can just generate a little bit different embed code and chat will be prefiled with logged user attributes,
- Send mail to user. You missed user chat, but still want to respond? The only option was just copy user email and write separte e-mail. Now you will be able to do it from live helper directly.
These three features are supported by one of my customers and he aggreed to share these enchancements with others. So you can say big thanks to him and me. P.s do not forget let others to know about Live Helper Chat.
P.s these features should go live in a weekend or so.
-
2013-04-21 19:21:06
Weekend over, some summary
Just after whole weekend of programming Live Helper Chat, here is list of things which were done during the weekend
Donations are really appereceated... Like Live Helper Chat is just a hobby :)


Voluntix
A platform to bring you volunteer experiences
Volunteer Events Portal
Volunteer
Experiences:
Please click on the location icon to show experiences closest to you.


Hello, Guest! This is the topic of the day.
Pesho wants to buy books
Pesho and his company Pesho
TM
decided to fill his collection of books. He wants following books:
Harry Potter
Hunger Games
The Da Vinci Code
The Cats
Address: Varna, street Ivan Ivanov 22
Telephone: 055533733
Email: peshobooks@abv.bg


AlloCore
About
AlloCore is a cross-platform suite of C++ components for building interactive multimedia tools and applications. In short, it provides a unified collection of easy-to-use classes and functions for system run-time control, media-related I/O interfaces and protocols, mathematical objects, graphics rendering, and spatial audio.
Source
Snapshots will be posted here...Bleeding edge development of AlloCore uses an internal repository that can be checked out via:
Resources
API (html)License
Allocore is provided under the following BSD-like license:Copyright (C) 2010. The Regents of the University of California (REGENTS). All Rights Reserved. Permission to use, copy, modify, distribute, and distribute modified versions of this software and its documentation without fee and without a signed licensing agreement, is hereby granted, provided that the above copyright notice, the list of contributors, this paragraph and the following two paragraphs appear in all copies, modifications, and distributions. IN NO EVENT SHALL REGENTS BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS DOCUMENTATION, EVEN IF REGENTS HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. REGENTS SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE AND ACCOMPANYING DOCUMENTATION, IF ANY, PROVIDED HEREUNDER IS PROVIDED "AS IS". REGENTS HAS NO OBLIGATION TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.


Using our knowledge from week 1 classes to recreate the wikipedia homepage
Using HTML and CSS to create a mock animal shelter website.
Day one practice with HTML. Very Cheesy.


--- layout: post title: "Galerie" permalink: /:title.html categories: - Photography published: true ---


{% extends "base_generic.html" %} {% block content %} Question: {{ output }} {% for elements in questions%} {{ elements }} {% endfor %} Don't see your question? Submit it {% endblock%}


Why HPX?¶
Current advances in high performance computing (HPC) continue to suffer from the issues plaguing parallel computation. These issues include, but are not limited to, ease of programming, inability to handle dynamically changing workloads, scalability, and efficient utilization of system resources. Emerging technological trends such as multi-core processors further highlight limitations of existing parallel computation models. To mitigate the aforementioned problems, it is necessary to rethink the approach to parallelization models. ParalleX contains mechanisms such as multi-threading, parcels, global name space support, percolation and local control objects (LCO). By design, ParalleX overcomes limitations of current models of parallelism by alleviating contention, latency, overhead and starvation. With ParalleX, it is further possible to increase performance by at least an order of magnitude on challenging parallel algorithms, e.g., dynamic directed graph algorithms and adaptive mesh refinement methods for astrophysics. An additional benefit of ParalleX is fine-grained control of power usage, enabling reductions in power consumption.
ParalleX—a new execution model for future architectures¶
ParalleX is a new parallel execution model that offers an alternative to the conventional computation models, such as message passing. ParalleX distinguishes itself by:
Split-phase transaction model
Message-driven
Distributed shared memory (not cache coherent)
Multi-threaded
Futures synchronization
Synchronization for anonymous producer-consumer scenarios
Percolation (pre-staging of task data)
The ParalleX model is intrinsically latency hiding, delivering an abundance of variable-grained parallelism within a hierarchical namespace environment. The goal of this innovative strategy is to enable future systems delivering very high efficiency, increased scalability and ease of programming. ParalleX can contribute to significant improvements in the design of all levels of computing systems and their usage from application algorithms and their programming languages to system architecture and hardware design together with their supporting compilers and operating system software.
What is HPX?¶
High Performance ParalleX (HPX) is the first runtime system implementation of the ParalleX execution model. The HPX runtime software package is a modular, feature-complete, and performance oriented representation of the ParalleX execution model targeted at conventional parallel computing architectures such as SMP nodes and commodity clusters. It is academically developed and freely available under an open source license. We provide HPX to the community for experimentation and application to achieve high efficiency and scalability for dynamic adaptive and irregular computational problems. HPX is a C++ library that supports a set of critical mechanisms for dynamic adaptive resource management and lightweight task scheduling within the context of a global address space. It is solidly based on many years of experience in writing highly parallel applications for HPC systems.
The two-decade success of the communicating sequential processes (CSP) execution model and its message passing interface (MPI) programming model has been seriously eroded by challenges of power, processor core complexity, multi-core sockets, and heterogeneous structures of GPUs. Both efficiency and scalability for some current (strong scaled) applications and future Exascale applications demand new techniques to expose new sources of algorithm parallelism and exploit unused resources through adaptive use of runtime information.
The ParalleX execution model replaces CSP to provide a new computing paradigm embodying the governing principles for organizing and conducting highly efficient scalable computations greatly exceeding the capabilities of today’s problems. HPX is the first practical, reliable, and performance-oriented runtime system incorporating the principal concepts of the ParalleX model publicly provided in open source release form.
HPX is designed by the STE||AR Group (Systems Technology, Emergent Parallelism, and Algorithm Research) at Louisiana State University (LSU)’s Center for Computation and Technology (CCT) to enable developers to exploit the full processing power of many-core systems with an unprecedented degree of parallelism. STE||AR is a research group focusing on system software solutions and scientific application development for hybrid and many-core hardware architectures.
What makes our systems slow?¶
Estimates say that we currently run our computers at way below 100% efficiency. The theoretical peak performance (usually measured in FLOPS—floating point operations per second) is much higher than any practical peak performance reached by any application. This is particularly true for highly parallel hardware. The more hardware parallelism we provide to an application, the better the application must scale in order to efficiently use all the resources of the machine. Roughly speaking, we distinguish two forms of scalability: strong scaling (see Amdahl’s Law) and weak scaling (see Gustafson’s Law). Strong scaling is defined as how the solution time varies with the number of processors for a fixed total problem size. It gives an estimate of how much faster can we solve a particular problem by throwing more resources at it. Weak scaling is defined as how the solution time varies with the number of processors for a fixed problem size per processor. In other words, it defines how much more data can we process by using more hardware resources.
In order to utilize as much hardware parallelism as possible an application must exhibit excellent strong and weak scaling characteristics, which requires a high percentage of work executed in parallel, i.e. using multiple threads of execution. Optimally, if you execute an application on a hardware resource with N processors it either runs N times faster or it can handle N times more data. Both cases imply 100% of the work is executed on all available processors in parallel. However, this is just a theoretical limit. Unfortunately, there are more things which limit scalability, mostly inherent to the hardware architectures and the programming models we use. We break these limitations into four fundamental factors which make our systems SLOW:
Starvation occurs when there is insufficient concurrent work available to maintain high utilization of all resources.
Latencies are imposed by the time-distance delay intrinsic to accessing remote resources and services.
Overhead is work required for the management of parallel actions and resources on the critical execution path which is not necessary in a sequential variant.
Waiting for contention resolution is the delay due to the lack of availability of oversubscribed shared resources.
Each of those four factors manifests itself in multiple and different ways; each of the hardware architectures and programming models expose specific forms. However the interesting part is that all of them are limiting the scalability of applications no matter what part of the hardware jungle we look at. Hand-helds, PCs, supercomputers, or the cloud, all suffer from the reign of the 4 horsemen: Starvation, Latency, Overhead, and Contention. This realization is very important as it allows us to derive the criteria for solutions to the scalability problem from first principles, it allows us to focus our analysis on very concrete patterns and measurable metrics. Moreover, any derived results will be applicable to a wide variety of targets.
Technology demands new response¶
Today’s computer systems are designed based on the initial ideas of John von Neumann, as published back in 1945, and later extended by the Harvard architecture. These ideas form the foundation, the execution model of computer systems we use currently. But apparently a new response is required in the light of the demands created by today’s technology.
So, what are the overarching objectives for designing systems allowing for applications to scale as they should? In our opinion, the main objectives are:
Performance: as mentioned, scalability and efficiency are the main criteria people are interested in
Fault tolerance: the low expected mean time between failures (MTBF) of future systems requires embracing faults, not trying to avoid them
Power: minimizing energy consumption is a must as it is one of the major cost factors today, even more so in the future
Generality: any system should be usable for a broad set of use cases
Programmability: for me as a programmer this is a very important objective, ensuring long term platform stability and portability
What needs to be done to meet those objectives, to make applications scale better on tomorrow’s architectures? Well, the answer is almost obvious: we need to devise a new execution model—a set of governing principles for the holistic design of future systems—targeted at minimizing the effect of the outlined SLOW factors. Everything we create for future systems, every design decision we make, every criteria we apply, has to be validated against this single, uniform metric. This includes changes in the hardware architecture we prevalently use today, and it certainly involves new ways of writing software, starting from the operating system, runtime system, compilers, and at the application level. However the key point is that all those layers have to be co-designed, they are interdependent and cannot be seen as separate facets. The systems we have today have been evolving for over 50 years now. All layers function in a certain way relying on the other layers to do so as well. However, we do not have the time to wait for a coherent system to evolve for another 50 years. The new paradigms are needed now—therefore, co-design is the key.
Governing principles applied while developing HPX¶
As it turn out, we do not have to start from scratch. Not everything has to be invented and designed anew. Many of the ideas needed to combat the 4 horsemen have already been had, often more than 30 years ago. All it takes is to gather them into a coherent approach. We’ll highlight some of the derived principles we think to be crucial for defeating SLOW. Some of those are focused on high-performance computing, others are more general.
Focus on latency hiding instead of latency avoidance¶
It is impossible to design a system exposing zero latencies. In an effort to come as close as possible to this goal many optimizations are mainly targeted towards minimizing latencies. Examples for this can be seen everywhere, for instance low latency network technologies like InfiniBand, caching memory hierarchies in all modern processors, the constant optimization of existing MPI implementations to reduce related latencies, or the data transfer latencies intrinsic to the way we use GPGPUs today. It is important to note, that existing latencies are often tightly related to some resource having to wait for the operation to be completed. At the same time it would be perfectly fine to do some other, unrelated work in the meantime, allowing the system to hide the latencies by filling the idle-time with useful work. Modern systems already employ similar techniques (pipelined instruction execution in the processor cores, asynchronous input/output operations, and many more). What we propose is to go beyond anything we know today and to make latency hiding an intrinsic concept of the operation of the whole system stack.
Embrace fine-grained parallelism instead of heavyweight Threads¶
If we plan to hide latencies even for very short operations, such as fetching the contents of a memory cell from main memory (if it is not already cached), we need to have very lightweight threads with extremely short context switching times, optimally executable within one cycle. Granted, for mainstream architectures this is not possible today (even if we already have special machines supporting this mode of operation, such as the Cray XMT). For conventional systems however, the smaller the overhead of a context switch and the finer the granularity of the threading system, the better will be the overall system utilization and its efficiency. For today’s architectures we already see a flurry of libraries providing exactly this type of functionality: non-pre-emptive, task-queue based parallelization solutions, such as Intel Threading Building Blocks (TBB), Microsoft Parallel Patterns Library (PPL), Cilk++, and many others. The possibility to suspend a current task if some preconditions for its execution are not met (such as waiting for I/O or the result of a different task), seamlessly switching to any other task which can continue, and to reschedule the initial task after the required result has been calculated, which makes the implementation of latency hiding almost trivial.
Rediscover constraint-based synchronization to replace global Barriers¶
The code we write today is riddled with implicit (and explicit) global barriers. By global barrier we mean the synchronization of the control flow between several (very often all) threads (when using OpenMP) or processes (MPI). For instance, an implicit global barrier is inserted after each loop parallelized using OpenMP as the system synchronizes the threads used to execute the different iterations in parallel. In MPI each of the communication steps imposes an explicit barrier onto the execution flow as (often all) nodes have to be synchronized. Each of those barriers acts as an eye of the needle the overall execution is forced to be squeezed through. Even minimal fluctuations in the execution times of the parallel threads (jobs) causes them to wait. Additionally it is often only one of the threads executing doing the actual reduce operation, which further impedes parallelism. A closer analysis of a couple of key algorithms used in science applications reveals that these global barriers are not always necessary. In many cases it is sufficient to synchronize a small subset of the threads. Any operation should proceed whenever the preconditions for its execution are met, and only those. Usually there is no need to wait for iterations of a loop to finish before you could continue calculating other things, all you need is to have those iterations done which were producing the required results for a particular next operation. Good bye global barriers, hello constraint based synchronization! People have been trying to build this type of computing (and even computers) already back in the 1970’s. The theory behind what they did is based on ideas around static and dynamic dataflow. There are certain attempts today to get back to those ideas and to incorporate them with modern architectures. For instance, a lot of work is being done in the area of constructing dataflow oriented execution trees. Our results show that employing dataflow techniques in combination with the other ideas, as outlined herein, considerably improves scalability for many problems.
Adaptive Locality Control instead of Static Data Distribution¶
While this principle seems to be a given for single desktop or laptop computers (the operating system is your friend), it is everything but ubiquitous on modern supercomputers, which are usually built from a large number of separate nodes (i.e. Beowulf clusters), tightly interconnected by a high bandwidth, low latency network. Today’s prevalent programming model for those is MPI which does not directly help with proper data distribution, leaving it to the programmer to decompose the data to all of the nodes the application is running on. There are a couple of specialized languages and programming environments based on PGAS (Partitioned Global Address Space) designed to overcome this limitation, such as Chapel, X10, UPC, or Fortress. However all systems based on PGAS rely on static data distribution. This works fine as long as such a static data distribution does not result in heterogeneous workload distributions or other resource utilization imbalances. In a distributed system these imbalances can be mitigated by migrating part of the application data to different localities (nodes). The only framework supporting (limited) migration today is Charm++. The first attempts towards solving related problem go back decades as well, a good example is the Linda coordination language. Nevertheless, none of the other mentioned systems support data migration today, which forces the users to either rely on static data distribution and live with the related performance hits or to implement everything themselves, which is very tedious and difficult. We believe that the only viable way to flexibly support dynamic and adaptive locality control is to provide a global, uniform address space to the applications, even on distributed systems.
Prefer moving work to the data over moving data to the work¶
For best performance it seems obvious to minimize the amount of bytes transferred from one part of the system to another. This is true on all levels. At the lowest level we try to take advantage of processor memory caches, thus minimizing memory latencies. Similarly, we try to amortize the data transfer time to and from GPGPUs as much as possible. At high levels we try to minimize data transfer between different nodes of a cluster or between different virtual machines on the cloud. Our experience (well, it’s almost common wisdom) show that the amount of bytes necessary to encode a certain operation is very often much smaller than the amount of bytes encoding the data the operation is performed upon. Nevertheless we still often transfer the data to a particular place where we execute the operation just to bring the data back to where it came from afterwards. As an example let me look at the way we usually write our applications for clusters using MPI. This programming model is all about data transfer between nodes. MPI is the prevalent programming model for clusters, it is fairly straightforward to understand and to use. Therefore, we often write the applications in a way accommodating this model, centered around data transfer. These applications usually work well for smaller problem sizes and for regular data structures. The larger the amount of data we have to churn and the more irregular the problem domain becomes, the worse are the overall machine utilization and the (strong) scaling characteristics. While it is not impossible to implement more dynamic, data driven, and asynchronous applications using MPI, it is overly difficult to do so. At the same time, if we look at applications preferring to execute the code close the locality where the data was placed, i.e. utilizing active messages (for instance based on Charm++), we see better asynchrony, simpler application codes, and improved scaling.
Favor message driven computation over message passing¶
Today’s prevalently used programming model on parallel (multi-node) systems is MPI. It is based on message passing (as the name implies), which means that the receiver has to be aware of a message about to come in. Both codes, the sender and the receiver, have to synchronize in order to perform the communication step. Even the newer, asynchronous interfaces require explicitly coding the algorithms around the required communication scheme. As a result, any more than trivial MPI application spends a considerable amount of time waiting for incoming messages, thus causing starvation and latencies to impede full resource utilization. The more complex and more dynamic the data structures and algorithms become, the larger are the adverse effects. The community has discovered message-driven and (data-driven) methods of implementing algorithms a long time ago, and systems such as Charm++ already have integrated active messages demonstrating the validity of the concept. Message driven computation allows sending messages without requiring the receiver to actively wait for them. Any incoming message is handled asynchronously and triggers the encoded action by passing along arguments and—possibly—continuations. HPX combines this scheme with work queue-based scheduling as described above, which allows the system to overlap almost completely any communication with useful work, thereby minimizing latencies.


White to play 42
pips White=194 Black=155
(CorrectAnswer= 0/0, ErrorScore= 0)
1. 23/21 13/9* eq: -0.061 49.17% (G:11.93% B:0.50%) 50.83% (G:15.40% B:0.54%) 2. 13/11 13/9* eq: -0.080 (-0.020) 48.98% (G:12.91% B:0.63%) 51.02% (G:17.34% B:0.79%) 3. 13/9* 9/7 eq: -0.083 (-0.022) 48.62% (G:12.35% B:0.57%) 51.38% (G:16.34% B:0.67%) 4. 20/18 13/9* eq: -0.100 (-0.039) 48.23% (G:12.22% B:0.57%) 51.77% (G:16.83% B:0.76%) 5. 13/9* 8/6 eq: -0.190 (-0.129) 46.60% (G:12.07% B:0.58%) 53.40% (G:19.30% B:0.94%)