COIN To use threads, one must rebuild the Coin Library (see below) To turn on Coin threadsafe, we set the Preprocessor variable COIN_THREADSAFE in building Coin. VSG/FEI Init is different with and without threads -- to use threads init the OpenInventor library with the following: SoWin::threadInit(myWindow); Both SoDB::threadInit(); ??? What does this do? is it necessary in either lib?
Main thread Does all OpenInventor calls -- builds tree Worker thread makes tiles -- all hi, then all med, then all low MrSID threads 1 or 2 threads Reads and decodes MrSID images, then transfers to cache Mutex tiles_rtv_class::mutex_stat_hi
Multithread users for hi-res tiles Worker thread Modifies graphics tree for individual tiles Main thread Traverses graphics tree Protection mechanism tiles_rtv_class::writelock(int ithread) tiles_rtv_class::writeunlock(int ithread) SoDB::writelock(); supplied by the COIN and VSG libraries SoDB::writeunlock();
Changes to rev 5.44 Add new methods map3d_index_class->is_elev_intersection(double roi_cen_north, double roi_cen_east, float roi_size_y, float roi_size_x); This is thread-safe and simpler -- used instead of calc_roi_intersections() Wired into some classes but not all map3d_index_class->dealloc_elev2(); No thread-safety effects Added code but did not implement thread-safe features map3d_index_class->get_elev_safe(double roi_cen_north, double roi_cen_east, float roi_size_y, float roi_size_x, int which_flag, int exec_smoothing_flag, int lowres_flag); This thread-vulnerable tasks into single method that can be more easily thread-safed Code kept but not wired into other classes Modified call sequence but did not implement new thread-safety map3d_index_class->get_elev_at_pt(double north, double east, int external_safe_flag, float &elev_a2);CLASS cache_class
Status variables are protected by a mutex so that only a single user can read/write them. The storage area is protected while writing by setting the stat array for that tile to 2 before starting to write and setting it to 1 on completion. That stat array is then checked that it is 1 before reading. The storage area is protected for later reading since it will never be deleted until it out of the display area.
CLASS map3d_index_class
Elevation Data
Elevation data is protected either by an internal mutex or by calling lock and unlock methods from calling classes that set the mutex. The internal mutex is used when only a single point elevation (or 2) is needed such done in mensuration. Locking and unlocking methods are called when elevations for a group of points are needed such as pinning a ground track to the terrain elevation.
Multithread users for hi-res tiles Elevation worker thread Add elevations to tiles Main thread Gets elevations for point or group of points such as mensuration Main thread Uses elevations to make false-color textures Main thread Uses elevations for LOS Protected data arrays data_a2 and data_a1 smoothing flags smooth_flags parms of current roi which tiles intersect (including method map3d_index->calc_roi_intersections()) Concept The lock method is called A block of elevation data is read The data is consumed -- transfered to cache, or a point-elevation is extracted, etc The unlock method can then be called since the data is no longer needed get_elev_at_pt(double north, double east, int external_safe_flag, float &elev_a2); map3d_index_class::elev_mutex_lock() Locks before a group of points is processed map3d_index_class::get_elev_at_pt(double north, double east, int external_safe_flag, float &elev_a2); Called with external_safe_flag=1 indicates that thread safing done external to class map3d_index_class::elev_mutex_unlock() Unlocks after group is processed kml_manager_inv_class.cpp(335): Pin line (multiple points) to ground before drawing it ladar_mm_manager_inv_class: Cull points very near ground level track_manager_class: Pin line (multiple points) to ground before drawing it OR map3d_index_class::get_elev_at_pt(double north, double east, int external_safe_flag, float &elev_a2); Called with external_safe_flag=0 Indicates thread safing is internal to this method kml_manager_inv_class: Display image just above the ground kml_manager_inv_class: Display single point? los_manager_inv_class: Display sensor loc (on tower) los_manager_inv_class: Draw a line from sensor to single loc mensuration_manager_inv_class: Find elev of picked pt or difference between 2 points track_manager_inv_class: Edit track -- Add or move a point get_elev(int which_flag, int exec_smoothing_flag, int lowres_flag); atr_los_class::read_a2_image: Read block of elevations, then transfer to local storage atr_los_class::make_mask(): Read block of elevations, then create shadow mask -- KEEPS LOCK FOR A LONG TIME image_rtv_class: No longer used -- do nothing map3d_index_class::make_low_res(): Make low-res tiles map3d_manager_inv_class: Make map tiles/make false-color textureCLASS tiles_rtv_class -- Hi-Res Tile Building Status Class tiles_rtv_class maintains the status of the building of hi-res tiles, med-res tiles and low-res tiles. Only the hi-res needs to be protected since that is the only one presently accessed by multiple threads.
Multithread users for hi-res tiles MrSID worker threads Add textures to tiles Elevation worker thread Add elevations to tiles Protected data Status array stat_hi Protected methods get_next_newtile_hi(int iFlag, int &iSpatialOut, int &iMemOut, double &north, double &east, float &height, float &width) init_newtile_hi(int *mem_flags) get_nnew_hi(int iMark) mark_tile_hi(int iSpatial, int iMark) get_Sep_status_hi(int iSep) Concept The next new tile is indentified and marked as under construction -- this is mutex protected Tile data (MrSID texture or elevation) is read and added to tile -- this does not need to be protected The data for the new tile is marked as complete -- this is mutex protectedLOS Calculations
LOS calculations use A2 elevations. These are protected using the elevation data safing described in map3d_index_class.It also produces masks. These are not protected -- it is very unlikely to produce a conflict and I think any corrupted data would be overwritten.
OLD THREAD SAFE MECHANISMS
OIV GLOBAL GL_Busy
Using classes los_manager_inv_class map3d_index_class map3d_manager_inv_class Worker thread -- map3d_manager_inv_class::update_tiles_next()Mutex Used to Safe MrSID Threads -- mutex_mrsid
Main thread -- IF no multithreading map3d_manager_inv_class::update_tiles() Worker thread map3d_manager_inv_class::update_tiles_next() MrSID worker threads map3d_manager_inv_class::update_mrsid_next(int iMrSID) Get next tile that needs MrSIDHISTORY
Multithreading is useful because maps can be much larger than can be accomodated in the hardware. Because of memory constraints with a 32-bit operating system and on the graphics card, only a portion of large map can displayed. Also, 3 different levels of detail are employed, high at the center and lower for areas further from the center. Maps are broken up into small tiles. When the user moves off the center portion of the map, new tiles around the new center of the map are read and processed and furthest away tiles are discarded to conserve memory. Worker threads speed up this process of creating new portions of the map.1. The first option is a worker thread that does all the tile updating. This option works with the VSG version of the viewer, since that library is thread-safe for different threads modifying objects and traversing the object tree. The option does not work reliably on the Coin-3d version of the viewer, since that library is not thread-safe.
2. The second option is a worker thread that does reading and decoding of MrSID-encoded high-resolution orthophoto texture. This process is CPU-intensive where most of the other tile-making processes are IO-intensive.
3. A third option is not truly multithreading, but has much the same effect. It allows the low-resolution tiles to be made by the main thread in the background -- only when the user interface is not active. Without an SSD and when data is being read for the first time, making the low-res tiles can much slower that making the high-res and medium-res tiles. This allows the user to view and manipulate the center portion of the image as the low-res edges are being constructed.
CURRENT MULTITHREADING OPTIONS
| Option | Tag | Variable | Default | |
|---|---|---|---|---|
| No of worker threads for MrSID read/decode | Map3d-Thread-MrSID | thread_flag_mrsid | 4 processors->2, 2 proc->1, 1 proc->0 | |
| Are tiles made (except MrSID) in worker thread? | Map3d-Thread-Tile | thread_flag_tile | VSG: yes, Coin: No (not threadsafe) | |
| Are low-res tiles made in the background? | Map3d-Thread-Low | thread_flag_low | VSG: No (covered by tiling thread), Coin: Yes |
Default sw_parms_class init for multithreading Map3d-Update 0 sw_parms_class init for single thread
I only implemented 2 threads. The second worker thread only constructs new tiles -- it reads the data from disk, does the processing necessary to determine smooth areas and constructs the OpenInventor object for each tile. The primary thead does everything else -- it handles user inputs, does calculations like track registration and LOS. When it needs to access map data, it requests that the worker thread pause so thread safety is assured. The worker thread is launched on program initialization and runs continually until map3dview is terminated. It continually tests whether new tiles are needed and, if so, makes them as fast as it can.
The Mercury OIV on Windows implements threading and claims to be completely thread-safe. It has a writelock/writeunlock that I put around the appropriate parts of the code to ensure that only 1 thread accesses objects at a time.
On the SGI I used Posix pthreads. The SGI OIV is not thread-safe as far as I know and I was not careful using mutexes, etc. I really dont care if this crashes occasionally on the SGI since I will only be doing debugging and not using the multi-thread option for real work here.
I implemented 3 options: Option 0 is the original technique with a single thread that makes the user wait until all tiles are updated before the display is updated.
Option 1 has a single thread, but updates in the background only when OIV is not busy. The trick here is to balance the tile updates with display refreshes. If you make too few tiles at a time, tile update is very slow. If you make too many, the tile update process monopolizes the processor making user response very slow. I have not found a decent balance between the two that significantly improves performance over the old method, so this option is not yet useful.
Option 2 is dual-thread, where the user can continue to interact with the display as the second thread updates the tiles. The trick here is not to allow the display to refresh too often -- if you allow it to refresh after each tile is created (my first thought), the huge number of refreshes makes the update impossibly long.  My current technique updates the med-res tiles in a single batch. It updates the low-res tiles in 2 batches -- the first just a few tiles around the med-res section, then all the rest in the second batch.
Before we set the variable, the code would crash, usually on moving after a few mouse clicks.
From email 5/5/2010 from Tom Fredrik Klaussen SIM My question: The second is multithreading. It appears from your documentation that multithreading is implemented but not turned on as a default. His answer: Rendering traversals are guaranteed to be thread-safe. There are other aspects of Coin that we know are thread-safe, but we make no guarantees in this regard. It needs to be enabled, since it incurs a small overhead. From SIM doc -- I think from Version 2: The support in Coin for using multiple threads in application programs and the Coin library itself, consists of two main features: Coin provides platform-independent thread-handling abstraction classes. These are classes that the application programmer can freely use in her application code to start new threads, control their execution, work with mutexes and do other tasks related to handling multiple threads. The classes in question are SbThread, SbMutex, SbStorage, SbBarrier, SbCondVar, SbFifo, SbThreadAutoLock, SbRWMutex, and SbTypedStorage. See their respective documentation for the detailed information. The classes fully hides the system-specific implementation, which is either done on top of native Win32 (if on Microsoft Windows), or over POSIX threads (on UNIX and UNIX-like systems). The other aspect of our multi-threading support is that Coin can be specially configured so that rendering traversals of the scene graph are done in a thread-safe manner. This means e.g. that it is possible to have Coin render the scene in parallel on multiple CPUs for multiple rendering pipes, to better take advantage of such high-end systems (like CAVE environments, for instance). Thread-safe render traversals are off by default, because there is a small overhead involved which would make rendering (very) slightly slower on single-threaded invocations. To get a Coin library built with thread-safe rendering, one must actively re-configure Coin and build a special, local version. For configure-based builds (UNIX and UNIX-like systems, or with Cygwin on Microsoft Windows) this is done with the option "--enable-threadsafe" to Autoconf configure. For how to change the configuration and re-build with Visual Studio, get in touch with us at<EMAIL_ADDRESS>There are some restrictions and other issues which it is important to be aware of: We do not yet provide any support for binding the multi-threaded rendering support into the SoQt / SoWin / etc GUI bindings, and neither do we provide bindings against any specific library that handles multi-pipe rendering. This means the application programmer will have to possess some expertise, and put in some effort, to be able to utilize multi-pipe rendering with Coin. Rendering traversals is currently the only operation which we publicly support to be thread-safe. There are other aspects of Coin that we know are thread-safe, like most other action traversals beside just rendering, but we make no guarantees in this regard. Be careful about using a separate thread for changing Coin structures versus what is used for the application's GUI event thread. We are aware of at least issues with Qt (and thereby SoQt), where you should not modify the scene graph in any way in a thread separate from the main Qt thread. This because it will trigger operations where Qt is not thread-safe.
GL_busy -- Used to request exclusive use of processors for multithreading = 0 Tiles made in main thread, so state can be ignored = 1 Worker thread makes tiles -- state is used = 2 Main thread request exclusive use of processors = 3 Worker threads suspended and grants exclusive to main thread 1. globals_inv_class.C: Defines GL_busy 2. map3d_manager_inv_class::make_scene_3d Init (0=no threads, 1=threads) Starts worker thread map3d_manager_inv_class::update_tiles_next Worker thread continuously checks GL_busy If GL_busy=2, this indicates that the main thread needs exclusive It set GL_busy=3 to indicate that it has paused and the main thread can proceed It checks GL_busy until GL_busy=1, indicating the main thread has finished It then resumes. 3. los_manager_inv_class::register_inv Requests exclusive so it can calc Line-of-sight 4. map3d_index_class::get_elev_at_pt Requests exclusive so it can get elev of single pt -- uses local store to minimize requests mensuration_manager_inv_class Calls to get elev for either info on single point or info on each end of line track_manager_inv_class Calls to get elev for track points ladar_mm_manager_inv_class::cull_near_ground Cull pts near ground 5. ch_manager_inv_class:: Requests exclusive so it can register group of frames
The updating process is probably very inefficient and can be streamlined. You want to update only those tiles done in specific pass (either med-res, or the inner ring of low-res or the outer ring of low-res).
Update map by emptying it and then adding in all tiles map3d_manager_inv_class::refresh() 1. Write lock tiles_rtv->writelock(0); 2. Turn on Notify -- allow IOV to sense that map is being updated cubesBase->enableNotify(TRUE); 3. Delete all tiles from the map cubesBase->removeAllChildren 4. Add all tiles cubesBase->addChild 2. Turn off Notify -- only want IOV to update map on explicit refresh cubesBase->enableNotify(FALSE); 4. Write unlock tiles_rtv->writeunlock(0);
Worker thread spawned in an infinite loop looking for tiles to update map3d_manager_inv_class::thread_method() Update any required tiles one-by-one map3d_manager_inv_class::update_tiles_next 1. Checks whether any med-res tiles need to be updated -- tiles_rtv->is_retile_med_required() 2. If med-res needs update, set up new tile structure for med-res tiles tiles_rtv->retile_med(); 3. Checks whether any low-res tiles need to be updated -- tiles_rtv->is_retile_low_required() 4. If low-res needs update, set up new tile structure for low-res tiles tiles_rtv->retile_low(); 5. If med-res needs update, Update 1 med-res tile map3d_manager_inv_class::make_newtile_med Else if low-res needs update, Update 1 low-res tile map3d_manager_inv_class::make_newtile_low Else Do nothing 6. Check if you should refresh (after med, first half of low, end of low) tiles_rtv->get_refresh_flag_med / get_refresh_flag_low Updata a single tile low/med 1. Read data from disk files into memory 2. Write lock tiles_rtv->writelock(1); -> SoDB::writelock(); 3. Update OIV object for tile map3d_manager_inv_class:: 2. Write unlock tiles_rtv->writeunlock(1); -> SoDB::writeunlock();
sensor_world from Visual Studio in debug mode Parms_Baghdad_DemifyJFD.s4d for map Map3d-Index-Loc Baghdad 33.564 44.046 33.086 44.623 33.3115 44.3496 35 10 int Map3d-Index-Find Baghdad C:/map3dview/rtv/Baghdad2_Tile37jfd dem_1m_a2_ .tif mrg 2 Map3d-Index-MrSID Baghdad baghdad122008 0 # MapID, MrSID prefix, flag (not used)
Single thread Map3d-Update 0 # 0=single thread as previous revs, 1=dont use this option, 2=threads Tiles-N 30 30 4 2 # Lod halfwidths -- xlo, ylo, med, hi (0-> 1 tile, -1-> no tiles) Large test Times med-res 25.2 hi-res 17.6 low-res 130.8 Additional thread for read/decode MrSID Map3d-Update 4 # 0=single thread as previous revs, 1=dont use this option, 2=threads Tiles-N 30 30 4 2 # Lod halfwidths -- xlo, ylo, med, hi (0-> 1 tile, -1-> no tiles) Large test Times med-res 38.3 19.1 19.5 hi-res 1.5 8.5 6.8 low-res 130.8
Single thread using SSD Map3d-Update 0 Tiles-N 30 30 4 2 # Lod halfwidths -- xlo, ylo, med, hi (0-> 1 tile, -1-> no tiles) Large test Times med-res 4.1 hi-res 9.8 low-res 26.5 Additional thread for read/decode MrSID using SSD Map3d-Update 4 # 0=single thread as previous revs, 1=dont use this option, 2=threads Tiles-N 30 30 4 2 # Lod halfwidths -- xlo, ylo, med, hi (0-> 1 tile, -1-> no tiles) Large test Times med-res 7.1 hi-res 2.4 low-res 26.3 2 Additional threads for read/decode MrSID Map3d-Update 20 Tiles-N 30 30 4 2 # Lod halfwidths -- xlo, ylo, med, hi (0-> 1 tile, -1-> no tiles) Large test Times med-res 7.0 7.3 hi-res 1.6 1.6 low-res 27.5 26.
void map3d_manager_inv_class::thread_method_mrsid() while (1) { // cout << " Thread to start pass " << icount << endl; rc = update_mrsid_next(); if (!rc) Sleep(20); // Windows takes msec } **************************************************************************** int map3d_manager_inv_class::update_mrsid_next() if ((tile_update_flag % 8) >= 4) { mutex_mrsid.lock(); } n_hi = tiles_rtv->get_nnew_hi(4); if (n_hi == 0) { if ((tile_update_flag % 8) >= 4) mutex_mrsid.unlock(); return(0); } // ************************************************ // Find texture for the single highest ranked remaining tile // ************************************************* tiles_rtv->get_next_newtile_hi(4, iSpatial,iSep, nc, ec, tile_height, tile_width); if ((tile_update_flag % 8) >= 4) mutex_mrsid.unlock(); get_texture_hi(iSep, nc, ec, tile_height, tile_width); if ((tile_update_flag % 8) >= 4) mutex_mrsid.lock(); if (tiles_rtv->mark_tile_hi(iSpatial, 8)) { store_tex_flags[iSep] = store_tex_flags[iSep] + 8; if (diag_flag > 0) cout << "iSpat=" << iSpatial << " iMem=" << iSep << " out of " << update_hi_n << " Texture mapped" << endl; } else { cout << "Texture mapped to mem index " << iSep << " ABORTED ***********" << endl; } if ((tile_update_flag % 8) >= 4) mutex_mrsid.unlock(); **************************************************************************** int map3d_manager_inv_class::get_texture_hi(int iTile, double north_cen, double east_cen, float height_tile, float width_tile) { map3d_index->set_roi_cen_hires_texture(north_cen, east_cen); map3d_index->set_roi_size_hires_texture(height_tile, width_tile); unsigned char *data_vhi = map3d_index->get_vhi(); int ny_tile = tiles_rtv->get_tiles_ny(); int nx_tile = tiles_rtv->get_tiles_nx(); int ratio_samples_vhi_to_a2 = map3d_index->get_ratio_vhi_to_elev(); for (int i=0; i<3*ratio_samples_vhi_to_a2*ratio_samples_vhi_to_a2*ny_tile*nx_tile; i++) { store_tex_hi[iTile][i] = data_vhi[i]; } return(1); }


The man in black led him to an ancient killing ground to make palaver. The gunslinger knew it immediately: a golgotha, place-of-the-skull. And bleached skulls stared blandly up at them—cattle, coyotes, deer, rabbits, bumbler. Here the alabaster xylophone of a hen pheasant killed as she fed; there the tiny, delicate bones of a mole, perhaps killed for pleasure by a wild dog.
The golgotha was a bowl indented into the descending slope of the mountain, and below, in easier altitudes, the gunslinger could see Joshua trees and scrub firs. The sky overhead was a softer blue than he had seen for a twelve-month, and there was an indefinable something that spoke of the sea in the not-too-great distance.
I am in the West, Cuthbert, he thought wonderingly. If this is not Mid-World, it’s close by.
The man in black sat on an ancient ironwood log. His boots were powdered white with dust and the uneasy bonemeal of this place. He had put his hood up again, but the gunslinger could see the square shape of his chin clearly, and the shading of his jaw.
The shadowed lips twitched in a smile. “Gather wood, gunslinger. This side of the mountains is gentle, but at this altitude, the cold still may put a knife in one’s belly.
“No you won’t. You can’t. But you can gather wood to remember your Isaac.”
The gunslinger had no understanding of the reference. He went wordlessly and gathered wood like a common cook’s boy. The pickings were slim. There was no devil-grass on this side and the ironwood would not burn. It had become stone. He returned finally with a large armload of likely sticks, powdered and dusted with disintegrated bone, as if dipped in flour. The sun had sunk beyond the highest Joshua trees and had taken on a reddish glow. It peered at them with baleful indifference.
“Excellent,” the man in black said. “How exceptional you are! How methodical! How resourceful! I salute you!” He giggled, and the gunslinger dropped the wood at his feet with a crash that ballooned up bone dust.
The man in black did not start or jump; he merely began laying the fire. The gunslinger watched, fascinated, as the ideogram (fresh, this time) took shape. When it was finished, it resembled a small and complex double chimney about two feet high. The man in black lifted his hand skyward, shaking back the voluminous sleeve from a tapered, handsome hand, and brought it down rapidly, index and pinky fingers forked out in the traditional sign of the evil eye. There was a blue flash of flame, and their fire was lighted.
“I have matches,” the man in black said jovially, “but I thought you might enjoy the magic. For a pretty, gunslinger. Now cook our dinner.”
The folds of his robe shivered, and the plucked and gutted carcass of a plump rabbit fell on the dirt.
The gunslinger spitted the rabbit wordlessly and roasted it. A savory smell drifted up as the sun went down. Purple shadows drifted hungrily over the bowl where the man in black had chosen to finally face him. The gunslinger felt hunger begin to rumble endlessly in his belly as the rabbit browned; but when the meat was cooked and its juices sealed in, he handed the entire skewer wordlessly to the man in black, rummaged in his own nearly flat knapsack, and withdrew the last of his jerky.

“Nevertheless,” the gunslinger said. There were tiny sores in his mouth, the result of vitamin deprivation, and the salt taste made him grin bitterly.
“Are you afraid of enchanted meat?”
“Yes indeed.”
The man in black slipped his hood back.
The gunslinger looked at him silently. In a way, the face that the hood had hidden was an uneasy disappointment. It was handsome and regular, with none of the marks and twists which indicate a man who has been through awesome times and has been privy to great secrets. His hair was black and of a ragged, matted length. His forehead was high, his eyes dark and brilliant. His nose was nondescript. The lips were full and sensual. His complexion was pallid, as was the gunslinger’s own.
The gunslinger said finally, “I expected an older man.”
“Why? I am nearly immortal, as are you, Roland—for now, at least. I could have taken a face with which you would have been more familiar, but I elected to show you the one I was—ah—born with. See, gunslinger, the sunset.”
The sun had departed already, and the western sky was filled with sullen furnace light.
“You won’t see another sunrise for what may seem a very long time,” the man in black said.
The gunslinger remembered the pit under the mountains and then looked at the sky, where the constellations sprawled in clockspring profusion.
“It doesn’t matter,” he said softly, “now.”
The man in black shuffled the cards with flying hands. The deck was huge, the designs on the back convoluted. “These are Tarot cards, gunslinger—of a sort. A mixture of the standard deck to which have been added a selection of my own development. Now watch carefully.”
“What will I watch?”
“I’m going to tell your future. Seven cards must be turned, one at a time, and placed in conjunction with the others. I’ve not done this since the days when Gilead stood and the ladies played at Points on the west lawn. And I suspect I’ve never read a tale such as yours.” Mockery was creeping into his voice again. “You are the world’s last adventurer. The last crusader. How that must please you, Roland! Yet you have no idea how close you stand to the Tower now, as you resume your quest. Worlds turn about your head.”
“What do you mean, resume? I never left off.”
At this the man in black laughed heartily, but would not say what he found so funny. “Read my fortune then,” Roland said harshly.
The first card was turned.
“The Hanged Man,” the man in black said. The darkness had given him back his hood. “Yet here, in conjunction with nothing else, it signifies strength, not death. You, gunslinger, are the Hanged Man, plodding ever onward toward your goal over the pits of Na’ar. You’ve already dropped one co-traveler into that pit, have you not?”
The gunslinger said nothing, and the second card was turned.
“The Sailor! Note the clear brow, the hairless cheeks, the wounded eyes. He drowns, gunslinger, and no one throws out the line. The boy Jake.”
The gunslinger winced, said nothing.
The third card was turned. A baboon stood grinningly astride a young man’s shoulder. The young man’s face was turned up, a grimace of stylized dread and horror on his features. Looking more closely, the gunslinger saw the baboon held a whip.
“The Prisoner,” the man in black said. The fire cast uneasy, flickering shadows over the face of the ridden man, making it seem to move and writhe in wordless terror. The gunslinger flicked his eyes away.
“A trifle upsetting, isn’t he?” the man in black said, and seemed on the verge of sniggering.
He turned the fourth card. A woman with a shawl over her head sat spinning at a wheel. To the gunslinger’s dazed eyes, she appeared to be smiling craftily and sobbing at the same time.
“The Lady of the Shadows,” the man in black remarked. “Does she look two-faced to you, gunslinger? She is. Two faces at least. She broke the blue plate!”
“What do you mean?”
“I don’t know.” And—in this case, at least—the gunslinger thought his adversary was telling the truth.
“Why are you showing me these?”
“Don’t ask!” the man in black said sharply, yet he smiled. “Don’t ask. Merely watch. Consider this only pointless ritual if it eases you and cools you to do so. Like church.”
He tittered and turned the fifth card.
A grinning reaper clutched a scythe with bony fingers. “Death,” the man in black said simply. “Yet not for you.”
The sixth card.
The gunslinger looked at it and felt a strange, crawling anticipation in his guts. The feeling was mixed with horror and joy, and the whole of the emotion was unnameable. It made him feel like throwing up and dancing at the same time.
“The Tower,” the man in black said softly. “Here is the Tower.”
The gunslinger’s card occupied the center of the pattern; each of the following four stood at one corner, like satellites circling a star.
“Where does that one go?” the gunslinger asked.
The man in black placed the Tower over the Hanged Man, covering it completely.
“What does that mean?” the gunslinger asked.
The man in black did not answer.
“What does that mean?” he asked raggedly.
The man in black did not answer.
“Goddamn you!”
No answer.
“Then be damned to you. What’s the seventh card?”
The man in black turned the seventh. A sun rose in a luminously blue sky. Cupids and sprites sported around it. Below the sun was a great red field upon which it shone. Roses or blood? The gunslinger could not tell. Perhaps, he thought, it’s both.
“The seventh card is Life,” the man in black said softly. “But not for you.”
“Where does it fit the pattern?”
“That is not for you to know now,” the man in black said. “Or for me to know. I’m not the great one you seek, Roland. I am merely his emissary.” He flipped the card carelessly into the dying fire. It charred, curled, and flashed to flame. The gunslinger felt his heart quail and turn icy in his chest.
“Sleep now,” the man in black said carelessly. “Perchance to dream and that sort of thing.”
“What my bullets won’t do, mayhap my hands will,” the gunslinger said. His legs coiled with savage, splendid suddenness, and he flew across the fire at the other, arms outstretched. The man in black, smiling, swelled in his vision and then retreated down a long and echoing corridor. The world filled with the sound of sardonic laughter, he was falling, dying, sleeping.
He dreamed.
The universe was void. Nothing moved. Nothing was.
The gunslinger drifted, bemused.
“Let’s have a little light,” the voice of the man in black said nonchalantly, and there was light. The gunslinger thought in a detached way that light was pretty good.
“Now darkness overhead with stars in it. Water down below.”
It happened. He drifted over endless seas. Above, the stars twinkled endlessly, yet he saw none of the constellations which had guided him across his long life.
“Land,” the man in black invited, and there was; it heaved itself out of the water in endless, galvanic convulsions. It was red, arid, cracked and glazed with sterility.
“Okay,” the man in black was saying. “That’s a start. Let’s have some plants. Trees. Grass and fields.”
There was. Dinosaurs rambled here and there, growling and whoofing and eating each other and getting stuck in bubbling, odiferous tarpits. Huge tropical rain-forests sprawled everywhere. Giant ferns waved at the sky with serrated leaves. Beetles with two heads crawled on some of them. All this the gunslinger saw. And yet he felt big.
“Now bring man,” the man in black said softly, but the gunslinger was falling . . . falling up. The horizon of this vast and fecund earth began to curve. Yes, they had all said it curved, his teacher Vannay had claimed it had been proved long before the world had moved on. But this—
Further and further, higher and higher. Continents took shape before his amazed eyes, and were obscured with clocksprings of clouds. The world’s atmosphere held it in a placental sac. And the sun, rising beyond the earth’s shoulder—
He cried out and threw an arm before his eyes.
“Let there be light!”
The voice no longer belonged to the man in black. It was gigantic, echoing. It filled space, and the spaces between space.
“Light!”
Falling, falling.
The sun shrank. A red planet stamped with canals whirled past him, two moons circling it furiously. Beyond this was a whirling belt of stones and a gigantic planet that seethed with gases, too huge to support itself, oblate in consequence. Further out was a ringed world that glittered like a precious gem within its engirdlement of icy spicules.
“Light! Let there be—”
Other worlds, one, two, three. Far beyond the last, one lonely ball of ice and rock twirled in dead darkness about a sun that glittered no brighter than a tarnished penny.
Beyond this, darkness.
“No,” the gunslinger said, and his word on it was flat and echoless in the black. It was darker than dark, blacker than black. Beside this, the darkest night of a man’s soul was as noonday, the darkness under the mountains a mere smudge on the face of Light. “No more. Please, no more now. No more—”
“LIGHT!”
“No more. No more, please—”
The stars themselves began to shrink. Whole nebulae drew together and became glowing smudges. The whole universe seemed to be drawing around him.
“Please no more no more no more—”
The voice of the man in black whispered silkily in his ear: “Then renege. Cast away all thoughts of the Tower. Go your way, gunslinger, and begin the long job of saving your soul.”
He gathered himself. Shaken and alone, enwrapt in the darkness, terrified of an ultimate meaning rushing at him, he gathered himself and uttered the final answer on that subject:
“NEVER!”
“THEN LET THERE BE LIGHT!”
And there was light, crashing in on him like a hammer, a great and primordial light. Consciousness had no chance of survival in that great glare, but before it perished, the gunslinger saw something clearly, something he believed to be of cosmic importance. He clutched it with agonized effort and then went deep, seeking refuge in himself before that light should blind his eyes and blast his sanity.
He fled the light and the knowledge the light implied, and so came back to himself. Even so do the rest of us; even so the best of us.
It was still night—whether the same or another, he had no immediate way of knowing. He pushed himself up from where his demon spring at the man in black had carried him and looked at the ironwood where Walter o’ Dim (as some along Roland’s way had named him) had been sitting. He was gone.
A great sense of despair flooded him—God, all that to do over again—and then the man in black said from behind him: “Over here, gunslinger.  You talk in your sleep.” He tittered.
The gunslinger got groggily to his knees and turned around. The fire had burned down to red embers and gray ashes, leaving the familiar decayed pattern of exhausted fuel. The man in black was seated next to it, smacking his lips with unlovely enthusiasm over the greasy remains of the rabbit.
“You did fairly well,” the man in black said. “I never could have sent that vision to your father. He would have come back drooling.”
“What was it?” the gunslinger asked. His words were blurred and shaky. He felt that if he tried to rise, his legs would buckle.
“The universe,” the man in black said carelessly. He burped and threw the bones into the fire where they first glistened and then blackened. The wind above the cup of the golgotha keened and moaned.
“Universe?” the gunslinger said blankly. It was a word with which he was unfamiliar. His first thought was that the other was speaking poetry.
“You want the Tower,” the man in black said. It seemed to be a question.
“Yes.”
“Well, you shan’t have it,” the man in black said, and smiled with bright cruelty. “No one cares in the counsels of the great if you pawn your soul or sell it outright, Roland. I have an idea of how close to the edge that last pushed you.
“I made your father and I broke him,” the man in black said grimly.  She bent beneath me like a willow . . . although (this may comfort you) she never broke. In any case it was written, and it was. I am the furthest minion of he who now rules the Dark Tower, and Earth has been given into that king’s red hand.”
“Red? Why do you say red?”
“Never mind. We’ll not speak of him, although you’ll learn more than you cared to if you press on.  This is not the beginning but the beginning’s end. You’d do well to remember that . . . but you never do.”
“I don’t understand.”
“No. You don’t. You never did. You never will. You have no imagination. You’re blind that way.”
“What did I see?” the gunslinger asked. “What did I see at the end? What was it?”
“What did it seem to be?”
The gunslinger was silent, thoughtful. He felt for his tobacco, but there was none. The man in black did not offer to refill his poke by either black magic or white. Later he might find more in his grow-bag, but later seemed very far away now.
“There was light,” the gunslinger said finally. “Great white light. And then—” He broke off and stared at the man in black. He was leaning forward, and an alien emotion was stamped on his face, writ too large for lies or denial. It was awe or wonder. Perhaps they were the same.
“You don’t know,” he said, and began to smile. “O great sorcerer who brings the dead to life. You don’t know. You’re a fake!”
“I know,” the man in black said. “But I don’t know . . . what.”
“White light,” the gunslinger repeated. “And then—a blade of grass. One single blade of grass that filled everything. And I was tiny. Infinitesimal.”
“Grass.” The man in black closed his eyes. His face looked drawn and haggard. “A blade of grass. Are you sure?”
“Yes.” The gunslinger frowned. “But it was purple.”
“Hear me now, Roland, son of Steven. Would you hear me?”
“Yes.”
And so the man in black began to speak.
The universe (he said) is the Great All, and offers a paradox too great for the finite mind to grasp. As the living brain cannot conceive of a nonliving brain—although it may think it can—the finite mind cannot grasp the infinite.
The prosaic fact of the universe’s existence alone defeats both the pragmatist and the romantic. There was a time, yet a hundred generations before the world moved on, when mankind had achieved enough technical and scientific prowess to chip a few splinters from the great stone pillar of reality. Even so, the false light of science (knowledge, if you like) shone in only a few developed countries. One company (or cabal) led the way in this regard; North Central Positronics, it called itself. Yet, despite a tremendous increase in available facts, there were remarkably few insights.
“Gunslinger, our many-times-great grandfathers conquered the-disease-which-rots, which they called cancer, almost conquered aging, walked on the moon—”
“I don’t believe that,” the gunslinger said flatly.
To this the man in black merely smiled and answered, “You needn’t. Yet it was so. They made or discovered a hundred other marvelous baubles. But this wealth of information produced little or no insight. There were no great odes written to the wonders of artificial insemination—having babies from frozen mansperm—or to the cars that ran on power from the sun. Few if any seemed to have grasped the truest principle of reality: new knowledge leads always to yet more awesome mysteries. Greater physiological knowledge of the brain makes the existence of the soul less possible yet more probable by the nature of the search. Do you see? Of course you don’t. You’ve reached the limits of your ability to comprehend. But never mind—that’s beside the point.”
“What is the point, then?”
“The greatest mystery the universe offers is not life but size. Size encompasses life, and the Tower encompasses size. The child, who is most at home with wonder, says: Daddy, what is above the sky? And the father says: The darkness of space. The child: What is beyond space? The father: The galaxy. The child: Beyond the galaxy? The father: Another galaxy. The child: Beyond the other galaxies? The father: No one knows.
“You see? Size defeats us. For the fish, the lake in which he lives is the universe. What does the fish think when he is jerked up by the mouth through the silver limits of existence and into a new universe where the air drowns him and the light is blue madness? Where huge bipeds with no gills stuff it into a suffocating box and cover it with wet weeds to die?
“Or one might take the tip of a pencil and magnify it. One reaches the point where a stunning realization strikes home: The pencil-tip is not solid; it is composed of atoms which whirl and revolve like a trillion demon planets. What seems solid to us is actually only a loose net held together by gravity. Viewed at their actual size, the distances between these atoms might become leagues, gulfs, aeons. The atoms themselves are composed of nuclei and revolving protons and electrons. One may step down further to subatomic particles. And then to what? Tachyons? Nothing? Of course not. Everything in the universe denies nothing; to suggest an ending is the one absurdity.
“If you fell outward to the limit of the universe, would you find a board fence and signs reading DEAD END? No. You might find something hard and rounded, as the chick must see the egg from the inside. And if you should peck through that shell (or find a door), what great and torrential light might shine through your opening at the end of space? Might you look through and discover our entire universe is but part of one atom on a blade of grass?  That existence rises not to one infinite but to an infinity of them?
“Perhaps you saw what place our universe plays in the scheme of things—as no more than an atom in a blade of grass. Could it be that everything we can perceive, from the microscopic virus to the distant Horsehead Nebula, is contained in one blade of grass that may have existed for only a single season in an alien time-flow? What if that blade should be cut off by a scythe? When it begins to die, would the rot seep into our own universe and our own lives, turning everything yellow and brown and desiccated? Perhaps it’s already begun to happen. We say the world has moved on; maybe we really mean that it has begun to dry up.
“Think how small such a concept of things makes us, gunslinger! If a God watches over it all, does He actually mete out justice for a race of gnats among an infinitude of races of gnats? Does His eye see the sparrow fall when the sparrow is less than a speck of hydrogen floating disconnected in the depth of space? And if He does see . . . what must the nature of such a God be? Where does He live? How is it possible to live beyond infinity?
“Imagine the sand of the Mohaine Desert, which you crossed to find me, and imagine a trillion universes—not worlds but universes—encapsulated in each grain of that desert; and within each universe an infinity of others. We tower over these universes from our pitiful grass vantage point; with one swing of your boot you may knock a billion billion worlds flying off into darkness, in a chain never to be completed.
“Size, gunslinger . . . size . . .
“Yet suppose further. Suppose that all worlds, all universes, met in a single nexus, a single pylon, a Tower. And within it, a stairway, perhaps rising to the Godhead itself. Would you dare climb to the top, gunslinger? Could it be that somewhere above all of endless reality, there exists a Room? . . .
“You dare not.”
And in the gunslinger’s mind, those words echoed: You dare not.
“Someone has dared,” the gunslinger said.
“Who would that be?”
“God,” the gunslinger said softly. His eyes gleamed.  Fear passed over the man in black’s bland face, as soft and dark as a buzzard’s wing. “And, furthermore, I don’t ask. It might be unwise.”
“Afraid of being struck dead?”
“Perhaps afraid of . . . an accounting.”
The man in black was silent for a while. The night was very long. The Milky Way sprawled above them in great splendor, yet terrifying in the emptiness between its burning lamps. The gunslinger wondered what he would feel if that inky sky should split open and let in a torrent of light.
“The fire,” he said. “I’m cold.”
“Build it up yourself,” said the man in black. “It’s the butler’s night off.”
The gunslinger drowsed awhile and awoke to see the man in black regarding him avidly, unhealthily.
“What are you staring at?” An old saying of Cort’s occurred to him.  He poked up the fire, ruining the precision of the ideogram. “I don’t like it.” He looked to the east to see if there was the beginning of light, but this night went on and on.
“You seek the light so soon.”
“I was made for light.”
“Ah, so you were! And so impolite of me to forget the fact! Yet we have much to discuss yet, you and I. For so has it been told to me by my king and master.”
“Who is this king?”
The man in black smiled. “Shall we tell the truth then, you and I? No more lies?”
“I thought we had been.”
But the man in black persisted as if Roland hadn’t spoken. “Shall there be truth between us, as two men? Not as friends, but as equals? There is an offer you will get rarely, Roland. Only equals speak the truth, that’s my thought on’t. Friends and lovers lie endlessly, caught in the web of regard. How tiresome!”
“Well, I wouldn’t want to tire you, so let us speak the truth.” He had never spoken less on this night. “Start by telling me what exactly you mean by glammer.”
“Why, enchantment, gunslinger! My king’s enchantment has prolonged this night and will prolong it until our palaver is done.”
“How long will that be?”
“Long. I can tell you no better. I do not know myself.” The man in black stood over the fire, and the glowing embers made patterns on his face. “Ask. I will tell you what I know. You have caught me. It is fair; I did not think you would. Yet your quest has only begun. Ask.  But before you meet him, you must first meet the Ageless Stranger.” The man in black smiled spitelessly. “You must slay him, gunslinger. Yet I think it is not what you wished to ask.”
“If you’ve never seen your king and master, how do you know him?”
“He comes to me in dreams. As a stripling he came to me, when I lived, poor and unknown, in a far land. A sheaf of centuries ago he imbued me with my duty and promised me my reward, although there were many errands in my youth and the days of my manhood, before my apotheosis. You are that apotheosis, gunslinger. You are my climax.” He tittered. “You see, someone has taken you seriously.”
“And this Stranger, does he have a name?”
“O, he is named.”
“And what is his name?”
“Legion,” the man in black said softly, and somewhere in the easterly darkness where the mountains lay, a rockslide punctuated his words and a puma screamed like a woman. The gunslinger shivered and the man in black flinched. “Yet I do not think that is what you wished to ask, either. It is not your nature to think so far ahead.”
The gunslinger knew the question; it had gnawed him all this night, and he thought, for years before. It trembled on his lips but he didn’t ask it . . . not yet.
“This Stranger is a minion of the Tower? Like yourself?”
“Yar. He darkles. He tincts. He is in all times. Yet there is one greater than he.”
“Who?”
“Ask me no more!” the man in black cried. His voice aspired to sternness and crumbled into beseechment. “I know not! I do not wish to know. To speak of the things in End-World is to speak of the ruination of one’s own soul.”
“And beyond the Ageless Stranger is the Tower and whatever the Tower contains?”
“Yes,” whispered the man in black. “But none of these things are what you wish to ask.”
True.
“All right,” the gunslinger said, and then asked the world’s oldest question. “Will I succeed?  You need killing.” His hands had dropped to the worn butts of his guns.
“Those do not open doors, gunslinger; those only close them forever.”
“Where must I go?”
“Start west. Go to the sea. Where the world ends is where you must begin. There was a man who gave you advice . . . the man you bested so long ago—”
“Yes, Cort,” the gunslinger interrupted impatiently.
“The advice was to wait. It was bad advice. For even then my plans against your father had proceeded.
“Then hear this: when you returned, Marten had gone west, to join the rebels. So all said, anyway, and so you believed. Yet he and a certain witch left you a trap and you fell into it. Good boy! And although Marten was long gone by then, there was a man who sometimes made you think of him, was there not? A man who affected the dress of a monk and the shaven head of a penitent—”
“Walter,” the gunslinger whispered. And although he had come so far in his musings, the bald truth still amazed him. “You. Marten never left at all.”
The man in black tittered.  Besides, all of that was long ago. Now comes the time of sharing.”
“You never left,” the gunslinger repeated, stunned. “You only changed.”
“Sit,” the man in black invited. “I’ll tell you stories, as many as you would hear. Your own stories, I think, will be much longer.”
“I don’t talk of myself,” the gunslinger muttered.
“Yet tonight you must. So that we may understand.”
“Understand what? My purpose? You know that. To find the Tower is my purpose. I’m sworn.”
“Not your purpose, gunslinger. Your mind. Your slow, prodding, tenacious mind. There has never been one quite like it, in all the history of the world. Perhaps in the history of creation.
“This is the time of speaking. This is the time of histories.”
“Then speak.”
The man in black shook the voluminous arm of his robe. A foil-wrapped package fell out and caught the dying embers in many reflective folds.
“Tobacco, gunslinger. Would you smoke?”
He had been able to resist the rabbit, but he could not resist this. He opened the foil with eager fingers. There was fine crumbled tobacco inside, and green leaves to wrap it in, amazingly moist. He had not seen such tobacco for ten years.
He rolled two cigarettes and bit the ends of each to release the flavor. He offered one to the man in black, who took it. Each of them took a burning twig from the fire.
The gunslinger lit his cigarette and drew the aromatic smoke deep into his lungs, closing his eyes to concentrate the senses. He blew out with long, slow satisfaction.
“Is it good?” the man in black inquired.
“Yes. Very good.”
“Enjoy it. It may be the last smoke for you in a very long time.”
The gunslinger took this impassively.
“Very well,” the man in black said. “To begin then:
“You must understand the Tower has always been, and there have always been boys who know of it and lust for it, more than power or riches or women . . . boys who look for the doors that lead to it . . .”
There was talk then, a night’s worth of talk and God alone knew how much more (or how much was true), but the gunslinger remembered little of it later . . . and to his oddly practical mind, little of it seemed to matter. The man in black told him again that he must go to the sea, which lay no more than twenty easy miles to the west, and there he would be invested with the power of drawing.
“But that’s not exactly right, either,” the man in black said, pitching his cigarette into the remains of the campfire. “No one wants to invest you with a power of any kind, gunslinger; it is simply in you, and I am compelled to tell you, partly because of the sacrifice of the boy, and partly because it is the law; the natural law of things. Water must run downhill, and you must be told. You will draw three, I understand . . . but I don’t really care, and I don’t really want to know.”
“The three,” the gunslinger murmured, thinking of the Oracle.
“And then the fun begins! But, by then, I’ll be long gone. Goodbye, gunslinger. My part is done now. The chain is still in your hands. ’Ware it doesn’t wrap itself around your neck.”
Compelled by something outside him, Roland said, “You have one more thing to say, don’t you?”
“Yes,” the man in black said, and he smiled at the gunslinger with his depthless eyes and stretched one of his hands out toward him. “Let there be light.”
And there was light, and this time the light was good.
Roland awoke by the ruins of the campfire to find himself ten years older. His black hair had thinned at the temples and there had gone the gray of cobwebs at the end of autumn. The lines in his face were deeper, his skin rougher.
The remains of the wood he had carried had turned to something like stone, and the man in black was a laughing skeleton in a rotting black robe, more bones in this place of bones, one more skull in this golgotha.
Or is it really you? he thought. I have my doubts, Walter o’ Dim . . . I have my doubts, Marten-that-was.
He stood up and looked around. Then, with a sudden quick gesture, he reached toward the remains of his companion of the night before (if it was indeed the remains of Walter), a night that had somehow lasted ten years. He broke off the grinning jawbone and jammed it carelessly into the left hip pocket of his jeans—a fitting enough replacement for the one lost under the mountains.
“How many lies did you tell me?” he asked. Many, he was sure, but what made them good lies was that they had been mixed with the truth.
The Tower. Somewhere ahead, it waited for him—the nexus of Time, the nexus of Size.
He began west again, his back set against the sunrise, heading toward the ocean, realizing that a great passage of his life had come and gone. “I loved you, Jake,” he said aloud.
The stiffness wore out of his body and he began to walk more rapidly. By that evening he had come to the end of the land. He sat on a beach which stretched left and right forever, deserted. The waves beat endlessly against the shore, pounding and pounding. The setting sun painted the water in a wide strip of fool’s gold.
There the gunslinger sat, his face turned up into the fading light. He dreamed his dreams and watched as the stars came out; his purpose did not flag, nor did his heart falter; his hair, finer now and gray at the temples, blew around his head, and the sandalwood-inlaid guns of his father lay smooth and deadly against his hips, and he was lonely but did not find loneliness in any way a bad or ignoble thing. The dark came down and the world moved on. The gunslinger waited for the time of the drawing and dreamed his long dreams of the Dark Tower, to which he would someday come at dusk and approach, winding his horn, to do some unimaginable final battle.
* For a fuller discussion of the Bullshit Factor, see On Writing, published by Scribner’s in 2000.
* Those bound by destiny.
* One example of this will probably serve for all. In the previously issued text of The Gunslinger, Farson is the name of a town. In later volumes, it somehow became the name of a man: the rebel John Farson, who engineers the fall of Gilead, the city-state where Roland spends his childhood.


FROM: MR.james obi ATTENION: MONEY TRANSFER / INVESTMENT PROPOSAL INTRODUCTION: Please I do not wish to invade your privacy by sending this mail to you. I am Mr james obi , a Civil Servant in the Federal Ministry of Health . I know this proposal will come to you as a surprise because we have not met physically or through correspondence. I got your contact from our chamber of commerce here in Nigeria and have no doubt about your ability to handle a huge sum of money. THE SUBJECT: My father Chief Solomon Dama (Now Late) was the Royal Head of my community , Eleme (an oil rich town) in Nigeria . My community produces 5.8% of the total crude oil in Nigeria and 0.5% of the Dollar value of each barrel is paid to my father as royalty by the Federal Government . My father was also the head and chairman of Eleme Special Oil Trust Fund . In his position as the royal head and chairman of the Oil Trust Fund , he made some money which he left for me as the only heir to inherit. The money is Twenty Five Million, Two Hundred Thousand US Dollars ($25.2M). This money originated from the accumulated royalties between 1976-1998. Due to poor banking system in Nigeria and political instability as a result of past military interventions , he deposited this money in a vault with an open beneficiary to a security company pending when he will finish arrangement to transfer it abroad. He was planning this when he suddenly died late last year of heart attack. THE PROPOSAL: Just before my father died , he called my attention to the money and charged me to look for a foreigner who will assist in the transfer /investment of the funds abroad . So I will be very grateful if you can accept to help me in this regard . I promise to give you 20% of the total funds transferred to your vital bank account as compensation for your assistance . Five percent (5%) would be set aside to take care of all expenses we may incure during transaction. To indicate your interest , contact me urgenly and confidentially for more information and the roles you will play in this business. All the legal documents concerning this money will be sent to you as soon as we reach a compromise.


URIs and Mappings
One feature that sets LinkML apart from frameworks such as JSON-Schema and UML is that fact that each element of your schema has a globally unique IRI/URI. This is somewhat hidden behind the scenes, so you can ignore this feature if you like, but it is also very easy to use this, which can provide benefits in terms of reusing and linking schemas, and working with the linked data stack.
background: URIs, IRIs, and CURIEs
URIs and IRIs are generalizations of URLs. URIs are used as identifiers in linked data standards and vocabularies.
For example, in schema.org, the URI http://schema.org/Person is the identifier for the Person concept.
URIs can be shortended as CURIEs (Compact URIs). Given a prefix declaration where we map schema
to http://schema.org/
, then we can use the CURIE schema:Person
to denote the person concept
prefixes
A typical header for a linkml schema may look like this:
id: https://w3id.org/linkml/examples/personinfo
name: personinfo
default_curi_maps:
- semweb_context
prefixes:
personinfo: https://w3id.org/linkml/examples/personinfo/
linkml: https://w3id.org/linkml/
schema: http://schema.org/
rdfs: http://www.w3.org/2000/01/rdf-schema#
prov: http://www.w3.org/ns/prov#
default_prefix: personinfo
emit_prefixes:
- rdf
- rdfs
- xsd
- skos
imports:
- linkml:types
The prefixes section contains a list of prefix expansions that can be used to specify CURIEs. Additionally, prefixmaps can be imported from prefixcommons.
With the above prefixmap, the CURIE schema:Person
will expand to http://schema.org/Person
class uri and slot uri
The two slots
can be used to declare URIs for classes and slots respectively. These are typically specified as CURIEs
classes:
Person:
is_a: NamedThing
description: >-
A person (alive, dead, undead, or fictional).
class_uri: schema:Person
...
slots:
id:
identifier: true
slot_uri: schema:identifier
name:
slot_uri: schema:name
...
When a JSON-LD context is generated for this schema, it will map json elements to their full linked data IRIs
If class and slot uris are omitted, then they are still generated behind the scenes, using the default_prefix slot at the schema level
For example, if Person did not declare a class_uri, then a CURIE personinfo:Person
would be used, which would expand to https://w3id.org/linkml/examples/personinfo/Person
mappings
You may wish to avoid committing to completely reusing a linked data concept, whilst wanting to retain a mapping. LinkML makes use of SKOS predicates as metamodel slots:
Example:
NamedThing:
description: >-
A generic grouping for any identifiable entity
slots:
- id
- name
- description
- image
close_mappings:
- schema:Thing
id_prefixes
The id_prefixes slot can be used to define a list of valid ID prefixes that instances of this class ought to have as part of their CURIE.
The order of the list matters since its a prioritized list with the ID prefix with the highest priority appearing at the top of the list.
For example, the biolink model defines a list of allowed id_prefixes for gene objects:
gene:
slots:
- id
- name
- symbol
- description
- synonym
- xref
exact_mappings:
- SO:0000704
- SIO:010035
- WIKIDATA:Q7187
id_prefixes:
- NCBIGene
- ENSEMBL
- HGNC
- UniProtKB
- MGI
- ZFIN
- dictyBase
- WB
- WormBase
- FlyBase
- FB
- RGD
- SGD
- PomBase
Here we define the entity class gene
to have a list of ID prefixes with NCBIGene
having the highest priority.
See Also
this notebook demonstrates some potential pitfalls of JSON-LD 1.0 with some forms of CURIEs


LearnOpenGL.com: Setup Gotchas on Ubuntu 16.04 and Code::Blocks
I began working through the fantastic OpenGL tutorials on learnopengl.com today and hit a couple of issues along the way.
Most of the issues I hit were caused by the fact that the tutorials are heavily biased towards Windows and Visual Studio while I’m using Ubuntu 16.04 and Code::Blocks.
Dependencies
The first issue I needed to resolve was the installation of the dependencies for GLFW. These can be installed using the following command:
sudo apt-get update && \
sudo apt-get -y install \
build-essential \
libx11-dev \
libxrandr-dev \
libxinerama-dev \
libxcursor-dev \
libgl1-mesa-dev
Linker
The next issue I faced was linking the GLFW library with my project in Code::Blocks.
After compiling the GLFW library from source (as per the tutorial) I decided to install GLFW from the Ubuntu repos (just to be sure I had a version that was compiled correctly).
sudo apt-get install libglfw3-dev
The tutorial recommended the following compiler flags to let the linker know where the required libraries are located:
-lGLEW -lglfw3 -lGL -lX11 -lpthread -lXrandr -lXi
But these flags gave me the following error:
cannot find -lglfw3
After a quick find
I found that the issue was the library name. It was not
called libglfw3 but was instead called libglfw:
$ find / -name *glfw* 2>/dev/null
...
/usr/lib/x86_64-linux-gnu/libglfw.so
...
To fix the issue you need to use -lglfw
in place of -lglfw3
.
NOTE: To set compiler linker flags in Code::Blocks you need to go to Settings > Compiler > Global Compiler Settings > Linker Settings > Other Linker Options
nullptr
I also had an issue with the nullptr
s used in the example code even though
my version of g++ is 5.4.0
. I was getting the following error:
error: 'nullptr' was not declared in this scope
To fix this I had to add the following compiler flag:
-std=gnu++0x
NOTE: To set compiler flags in Code::Blocks you need to go to Settings > Compiler > Global Compiler Settings > Compiler Settings > Other Compiler Options
std::cout
The final issue I had was with std::cout and std::endl. I was getting the following errors:
error: 'cout' is not a member of 'std'
error: 'endl' is not a member of 'std'
In order to fix this I needed to include iostream
.
#include <iostream>


Hamilton Twp. Free Public Library
1 Justice Samuel A. Alito Jr. Way
Hamilton Twp
,
New Jersey
08619
United States
| Mon.-Thurs. | 9:00am-8:30pm |
| Fri. & Sat. | 9:00am-5:00pm |
| Sunday | Closed |


Success!
Chatbot Administration
This page is used for different chatbot admin activities
{% extends "layout.html" %} {% block content %}
Please upload your intent file
Tip: Use the Control or the Shift key to select multiple files.
Start training your system by cliccking the button
{##}
Please upload your configration or training config file
Tip: Use the Control or the Shift key to select multiple files.


Multi Browser Opener
Configure
Quit
Do you want to add another account?
#
account-name
account-name
https://calendar.google.com/calendar/render
Open
Toggle Dropdown
Open Window
Edit This Account
Up
Down
Remove This Account


You're on the list! We'll be in touch.
We're still in early stages, and we'll let you know once we're ready for more beta testers.


Typically WINDOWS only
This method is far less known and not really usable on platforms other than Windows. It is possible to send a file with arbitrary code to any PHP script, and include the temporary file that the PHP engine has created.
if 'file_uploads = on' is set in the PHP configuration file, then PHP will accept a file upload post to any PHP file
Pros and cons
Good news is that it's common for a PHP script to have access to the directory (see upload_tmp_dir in php.ini) where the temporary files are created (i.e. can include files from this directory). On the default PHP installation the upload_tmp_dir is not set - in this case either /tmp on Linux-based systems or C:\Windows\Temp on Windows are used.
The bad news is that the name of the temporary file is random, which renders this method unusable in most cases - it is fully exploitable on Windows and exploitable in some cases on other systems.
On Windows
Only the lower 16 bits of the uUnique parameter are used. This limits GetTempFileName to a maximum of 65,535 unique file names if the lpPathName and lpPrefixStringparameters remain the same.
Only 65k unique names makes a brute force possible, and using current system time (number of milliseconds) makes things even simpler.
On GNU/Linux
This function, depending on the way glibc is compiled, uses either (in pseudocode; variables are uint64_t):
1. random_value = (seed += time() ^ PID)
2. random_value = (seed += (gettimeofday().sec << 32 | gettimeofday().usec) ^ PID)
3. random_value = (seed += rdtsc ^ PID)
The random_value is later written as 6 digits of k=62 (A-Za-z0-9 charset) numeric system, and appended to the "/tmp/php" prefix (unless another directory is set), e.g. /tmp/phpUs7MxA. From the aforementioned random_value generation methods, the 3rd is most commonly used nowadays. According to initial tests I made, the random value is unpredictable enough to be considered safe against remote attacker.
FindFirstFile 'quirk'
Allows using masks (<< as * and > as ?) in LFI paths on Windows. Thanks to this, one can form an include path like this:
http://site/vuln.php?inc=c:\windows\temp\php<<
Commonly there are other files with "php" prefix in this directory, so narrowing the mask might be required. In this case it's best to choose the widest possible mask that does not include any file (e.g. php1<< or phpA<<, in worse case php11<<, etc) and send upload-packets until the PHP engine will create a temporary file that will match the mask.
Assumptions needed for Linux exploitation
exploitation path is possible only in these specific cases:
When the tester is able to list the files in the upload (/tmp) directory (presumably by another PHP script) - this would require a race condition method where the tester uploads one file to a slow script (i.e. a script that runs for a relatively long time) and lists the files in the upload directory in another script, acquires the file temporary file name, and fires the LFI exploit.
When the tester is able to list the $_FILES array in a script - this requires an even more tight race condition and is not always possible, since the attacker must obtain the $_FILES array content during upload and send another packet before the previous script ends and the temporary file is removed. This is not possible in case of buffering of PHP output (e.g. if mod_gzip is enabled in Apache).