Apply private to some "no doc"ed symbols in the Random module

And renamed these private symbols to lose the RandomPrivate_ prefix.

/*
 * Copyright 2004-2015 Cray Inc.
 * Other additional copyright holders may be indicated within.
 *
 * The entirety of this work is licensed under the Apache License,
 * Version 2.0 (the "License"); you may not use this file except
 * in compliance with the License.
 *
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

/*
   Support for pseudorandom number generation

   This module defines an abstraction for a stream of pseudorandom
   numbers, :class:`RandomStream`, supporting methods to get the next
   random number in the stream (:proc:`.getNext`), to fast-forward to
   a specific value in the stream (:proc:`.skipToNth` and
   :proc:`.getNth`), or to fill an array with random numbers in
   parallel (:proc:`~Random.RandomStream.fillRandom`).  The module
   also provides a standalone convenience function, :proc:`fillRandom`
   that can be used to fill an array with random numbers in parallel
   without manually creating a :class:`RandomStream` object.

   The current pseudorandom number generator (PRNG) implemented by
   this module uses the algorithm from the NAS Parallel Benchmarks
   (NPB, available at: http://www.nas.nasa.gov/publications/npb.html),
   which can be used to generate random values of type `real(64)`,
   `imag(64)`, and `complex(128)`.  The longer-term intention is to
   add knobs to select between a menu of additional PRNG algorithms
   (such as the Mersenne twister) and element types (such as integer
   types and other floating point widths).

   Paraphrasing the comments from the NPB reference implementation:

     This generator returns uniform pseudorandom real values in the
     range (0, 1) by using the linear congruential generator

       `x_{k+1} = a x_k  (mod 2**46)`

     where 0 < x_k < 2**46 and 0 < a < 2**46.  This scheme
     generates 2**44 numbers before repeating.  The generated values
     are normalized to be between 0 and 1, i.e., 2**(-46) * x_k.

     This generator should produce the same results on any computer
     with at least 48 mantissa bits for `real(64)` data.

   Here is a list of currently open issues (TODOs) for this module:

   1. This module is currently restricted to generating `real(64)`,
   `imag(64)`, and `complex(128)` complex values using a single PRNG
   algorithm.  As noted above, we would like to extend this support to
   include other algorithms and primitive types over time.

   2. We plan to support general serial and parallel iterator methods
   on :class:`RandomStream`; however, providing the full suite of
   iterators is not possible with our current parallel iterator
   framework.  Specifically, if :class:`RandomStream` is a follower in
   a zippered iteration context, there is no way for it to update the
   total number of random numbers generated in a safe/sane/coordinated
   way.  We are exploring a revised leader-follower iterator framework
   that would support this idiom (and other cursor-based ones).  With
   Chapel's recent support for standalone parallel iterators, one
   could define a standalone parallel iterator for
   :class:`RandomStream`, but this effort has not yet been taken on.

   3. If no seed is provided by the user, one is chosen based on the
   current time in microseconds, allowing for some degree of
   pseudorandomness in seed selection.  The intent of
   :record:`SeedGenerators` is to provide a menu of other options
   for initializing the random stream seed, but only one option is
   implemented at present.

*/
module Random {

  /*
     Note for developers on "private" symbols:

     It is the intent that once Chapel supports a notion of 'private'
     symbols, everything prefixed with RandomPrivate will be made
     private to this module and everything prefixed with
     'RandomStreamPrivate_' will be made private to the RandomStream
     class.
  */

  /* Internal TODOs for developers:

     - should RandomStream be parameterized by the type to return?
       it seems odd that currently getNext() and getNth() return
       reals always...
  */

/*
   An instance of :record:`SeedGenerators` that provides a
   convenient means of generating seeds when the user does not wish to
   specify one manually.
*/
const SeedGenerator: SeedGenerators;

// CHPLDOC FIXME: Respect ordering of things in the file

/*
  Provides methods to help generate seeds when the user doesn't want
  to create one.  It currently only supports one such method, but the
  intention is to add more over time.

  .. note::
          Once Chapel supports static class methods,
          :const:`SeedGenerator` and :record:`SeedGenerators` should
          be combined into a single record type with static methods).

*/

record SeedGenerators {
  /*
    Generate a seed based on the current time in microseconds as
    reported by :proc:`Time.getCurrentTime`, ensuring that it
    meets the PRNG's requirements.
  */
  proc currentTime: int(64) {
    use Time;
    const seed: int(64) = getCurrentTime(unit=TimeUnits.microseconds):int(64);
    return (if seed % 2 == 0 then seed + 1 else seed) % (1:int(64) << 46);
  }
};

// CHPLDOC FEEDBACK: If easy, I'd suggest either deprecating the
// :arg <type> <name>: form or else switching the order to
// :arg <name> <type>: as it's an easy trap to simply type the same
// argument as in the function's signature.

// CHPLDOC FIXME: if the first line below is shifted one character to the
// left, it ends up being rendered like a method

// CHPLDOC FIXME: if the first line below is shifted one character to the
// right, it ends up causing a warning (promoted to error) in the .rst
// file.  It'd be preferable to have it declare the issue in terms of
// the .chpl line numbers.
//

/*
  Fill an array of `real(64)`, `imag(64)`, or `complex(128)` elements
  with pseudorandom values in parallel using a new
  :class:`RandomStream` created specifically for this call.  The
  first `arr.size` values from the stream will be assigned to the
  array's elements in row-major order for `real` and `imag` elements.
  For `complex` elements, consecutive pairs of random numbers are
  assigned to the real and imaginary components, respectively.  The
  parallelization strategy is determined by the array.

  :arg arr: The array to be filled, where T is `real(64)`, `imag(64)`, or `complex(128)`.
  :type arr: [] T

  :arg seed: The seed to use for the PRNG.  Defaults to :proc:`SeedGenerator.currentTime <SeedGenerators.currentTime>`.
  :type seed: int(64)
*/

proc fillRandom(arr: [], seed: int(64) = SeedGenerator.currentTime)
  where (arr.eltType == real || arr.eltType == imag || arr.eltType == complex) {
  var randNums = new RandomStream(seed, eltType=arr.eltType, parSafe=false);
  randNums.fillRandom(arr);
  delete randNums;
}

pragma "no doc"
proc fillRandom(arr: [], seed: int(64) = SeedGenerator.currentTime) {
  compilerError("Random.fillRandom is only defined for real(64), imag(64), and complex(128) arrays");
}

/*
  Models a stream of pseudorandom numbers.  See the module-level
  notes for :mod:`Random` for details on the PRNG used.
*/
class RandomStream {
  /*
    Specifies the type of value generated by the RandomStream.
    Currently, only `real(64)`, `imag(64)`, and `complex(128)` are
    supported.
  */
  type eltType = real(64);

  /*
    Indicates whether or not the RandomStream needs to be
    parallel-safe by default.  If multiple tasks interact with it in
    an uncoordinated fashion, this must be set to `true`.  If it will
    only be called from a single task, or if only one task will call
    into it at a time, setting to `false` will reduce overhead related
    to ensuring mutual exclusion.
  */
  param parSafe: bool = true;

  /*
    The seed value for the PRNG.  It must be an odd integer in the
    range (1, 2**46).
  */
  const seed: int(64);

  /*
    Constructs a new stream of random numbers using the specified seed
    and parallel safety.  Ensures that the seed value meets the PRNG's
    constraints.

    :arg seed: The seed to use for the PRNG.  Defaults to :proc:`SeedGenerator.currentTime <SeedGenerators.currentTime>`..
    :type seed: int(64)

    :arg parSafe: The parallel safety setting.  Defaults to `true`.
    :type parSafe: bool

    :arg eltType: The element type to be generated.  Defaults to `real(64)`.
    :type eltType: type
  */
  proc RandomStream(seed: int(64) = SeedGenerator.currentTime,
                    param parSafe: bool = true,
                    type eltType = real(64)) {
    if seed % 2 == 0 || seed < 1 || seed > 1:int(64)<<46 then
      halt("RandomStream seed must be an odd integer between 0 and 2**46");
    this.seed = seed;
    RandomStreamPrivate_cursor = seed;
    RandomStreamPrivate_count = 1;
  }

  pragma "no doc"
  proc RandomStreamPrivate_getNext_noLock() {
    if (eltType == complex) {
      RandomStreamPrivate_count += 2;
    } else {
      RandomStreamPrivate_count += 1;
    }
    return randlc(eltType, RandomStreamPrivate_cursor);
  }

  pragma "no doc"
  proc RandomStreamPrivate_skipToNth_noLock(in n: integral) {
    if eltType == complex then n = n*2 - 1;
    RandomStreamPrivate_count = n;
    RandomStreamPrivate_cursor = randlc_skipto(seed, n);
  }

  /*
    Returns the next value in the random stream.

    :returns: The next value in the random stream as type :type:`eltType`.
   */
  proc getNext(): eltType {
    if parSafe then
      RandomStreamPrivate_lock$ = true;
    const result = RandomStreamPrivate_getNext_noLock();
    if parSafe then
      RandomStreamPrivate_lock$;
    return result;
  }

  /*
    Advances/rewinds the stream to the `n`-th value in the sequence.

    :arg n: The position in the stream to skip to.  Must be non-negative.
    :type n: integral
   */

  proc skipToNth(n: integral) {
    if n <= 0 then
      halt("RandomStream.skipToNth(n) called with non-positive 'n' value", n);
    if parSafe then
      RandomStreamPrivate_lock$ = true;
    RandomStreamPrivate_skipToNth_noLock(n);
    if parSafe then
      RandomStreamPrivate_lock$;
  }

  /*
    Advance/rewind the stream to the `n`-th value and return it
    (advancing the stream by one).  This is equivalent to
    :proc:`skipToNth()` followed by :proc:`getNext()`.

    :arg n: The position in the stream to skip to.  Must be non-negative.
    :type n: integral

    :returns: The `n`-th value in the random stream as type :type:`eltType`.
  */

  proc getNth(n: integral): eltType {
    if (n <= 0) then
      halt("RandomStream.getNth(n) called with non-positive 'n' value", n);
    if parSafe then
      RandomStreamPrivate_lock$ = true;
    RandomStreamPrivate_skipToNth_noLock(n);
    const result = RandomStreamPrivate_getNext_noLock();
    if parSafe then
      RandomStreamPrivate_lock$;
    return result;
  }

  /*
    Fill the argument array with pseudorandom values.  This method is
    identical to the standalone :proc:`fillRandom` procedure,
    except that it consumes random values from the
    :class:`RandomStream` object on which it's invoked rather
    than creating a new stream for the purpose of the call.

    :arg arr: The array to be filled
    :type arr: [] :type:`eltType`
  */
  proc fillRandom(arr: [] eltType) {
    forall (x, r) in zip(arr, iterate(arr.domain, arr.eltType)) do
      x = r;
  }

  pragma "no doc"
  proc fillRandom(arr: []) {
    compilerError("RandomStream(eltType=", typeToString(eltType), ") can only be used to fill arrays of ", typeToString(eltType));
  }

  pragma "no doc"
  proc iterate(D: domain, type resultType=real) {
    if parSafe then
      RandomStreamPrivate_lock$ = true;
    const start = RandomStreamPrivate_count;
    RandomStreamPrivate_count += D.numIndices.safeCast(int(64));
    RandomStreamPrivate_skipToNth_noLock(RandomStreamPrivate_count);
    if parSafe then
      RandomStreamPrivate_lock$;
    return iterate(resultType, D, seed, start);
  }

  pragma "no doc"
  proc writeThis(f: Writer) {
    f <~> "RandomStream(eltType=";
    f <~> typeToString(eltType);
    f <~> ", parSafe=";
    f <~> parSafe;
    f <~> ", seed=";
    f <~> seed;
    f <~> ")";
  }

  ///////////////////////////////////////////////////////////// CLASS PRIVATE //
  //
  // It is the intent that once Chapel supports the notion of
  // 'private', everything in this class declared below this line will
  // be made private to this class.
  //

  pragma "no doc"
  var RandomStreamPrivate_lock$: sync bool;
  pragma "no doc"
  var RandomStreamPrivate_cursor: real = seed;
  pragma "no doc"
  var RandomStreamPrivate_count: int(64) = 1;
}

////////////////////////////////////////////////////////////// MODULE PRIVATE //
//
// It is the intent that once Chapel supports the notion of 'private',
// everything declared below this line will be made private to this
// module.
//

//
// NPB-defined constants for linear congruential generator
//
pragma "no doc"
private const r23   = 0.5**23,
              t23   = 2.0**23,
              r46   = 0.5**46,
              t46   = 2.0**46,
              arand = 1220703125.0; // TODO: Is arand something that a
                                    // user might want to set on a
                                    // case-by-case basis?

//
// NPB-defined randlc routine
//
pragma "no doc"
private proc randlc(inout x: real, a: real = arand) {
  var t1 = r23 * a;
  const a1 = floor(t1),
    a2 = a - t23 * a1;
  t1 = r23 * x;
  const x1 = floor(t1),
    x2 = x - t23 * x1;
  t1 = a1 * x2 + a2 * x1;
  const t2 = floor(r23 * t1),
    z  = t1 - t23 * t2,
    t3 = t23 * z + a2 * x2,
    t4 = floor(r46 * t3),
    x3 = t3 - t46 * t4;
  x = x3;
  return r46 * x3;
}

// Wrapper that takes a result type (two calls for complex types)
pragma "no doc"
private proc randlc(type resultType, inout x: real) {
  if resultType == complex then
    return (randlc(x), randlc(x)):complex;
  else
    if resultType == imag then
      //
      // BLC: I thought that casting real to imag did this automatically?
      //
      return _r2i(randlc(x));
    else
      return randlc(x);
 }

//
// Return a value for the cursor so that the next call to randlc will
// return the same value as the nth call to randlc
//
pragma "no doc"
private proc randlc_skipto(seed: int(64), in n: integral): real {
  var cursor = seed:real;
  n -= 1;
  var t = arand;
  arand;
  while (n != 0) {
    const i = n / 2;
    if (2 * i != n) then
      randlc(cursor, t);
    if i == 0 then
      break;
    else
      n = i;
    randlc(t, t);
    n = i;
  }
  return cursor;
}

//
// iterate over outer ranges in tuple of ranges
//
pragma "no doc"
private iter outer(ranges, param dim: int = 1) {
  if dim + 1 == ranges.size {
    for i in ranges(dim) do
      yield (i,);
  } else if dim + 1 < ranges.size {
    for i in ranges(dim) do
      for j in outer(ranges, dim+1) do
        yield (i, (...j));
  } else {
    yield 0; // 1D case is a noop
  }
}

//
// RandomStream iterator implementation
//
pragma "no doc"
private iter iterate(type resultType, D: domain, seed: int(64),
                     start: int(64)) {
  var cursor = randlc_skipto(seed, start);
  for i in D do
    yield randlc(resultType, cursor);
}

pragma "no doc"
private iter iterate(type resultType, D: domain, seed: int(64),
                     start: int(64), param tag: iterKind)
      where tag == iterKind.leader {
  for block in D._value.these(tag=iterKind.leader) do
    yield block;
}

pragma "no doc"
iter iterate(type resultType, D: domain, seed: int(64),
             start: int(64), param tag: iterKind, followThis)
      where tag == iterKind.follower {
  param multiplier = if resultType == complex then 2 else 1;
  const ZD = computeZeroBasedDomain(D);
  const innerRange = followThis(ZD.rank);
  var cursor: real;
  for outer in outer(followThis) {
    var myStart = start;
    if ZD.rank > 1 then
      myStart += multiplier * ZD.indexOrder(((...outer), innerRange.low)).safeCast(int(64));
    else
      myStart += multiplier * ZD.indexOrder(innerRange.low).safeCast(int(64));
    if !innerRange.stridable {
      cursor = randlc_skipto(seed, myStart);
      for i in innerRange do
        yield randlc(resultType, cursor);
    } else {
      myStart -= innerRange.low.safeCast(int(64));
      for i in innerRange {
        cursor = randlc_skipto(seed, myStart + i.safeCast(int(64)) * multiplier);
        yield randlc(resultType, cursor);
      }
    }
  }
}

} // close module Random

/*
 * Copyright 2004-2015 Cray Inc.
 * Other additional copyright holders may be indicated within.
 *
 * The entirety of this work is licensed under the Apache License,
 * Version 2.0 (the "License"); you may not use this file except
 * in compliance with the License.
 *
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

/*
   Support for pseudorandom number generation

   This module defines an abstraction for a stream of pseudorandom
   numbers, :class:`RandomStream`, supporting methods to get the next
   random number in the stream (:proc:`.getNext`), to fast-forward to
   a specific value in the stream (:proc:`.skipToNth` and
   :proc:`.getNth`), or to fill an array with random numbers in
   parallel (:proc:`~Random.RandomStream.fillRandom`).  The module
   also provides a standalone convenience function, :proc:`fillRandom`
   that can be used to fill an array with random numbers in parallel
   without manually creating a :class:`RandomStream` object.

   The current pseudorandom number generator (PRNG) implemented by
   this module uses the algorithm from the NAS Parallel Benchmarks
   (NPB, available at: http://www.nas.nasa.gov/publications/npb.html),
   which can be used to generate random values of type `real(64)`,
   `imag(64)`, and `complex(128)`.  The longer-term intention is to
   add knobs to select between a menu of additional PRNG algorithms
   (such as the Mersenne twister) and element types (such as integer
   types and other floating point widths).

   Paraphrasing the comments from the NPB reference implementation:

     This generator returns uniform pseudorandom real values in the
     range (0, 1) by using the linear congruential generator

       `x_{k+1} = a x_k  (mod 2**46)`

     where 0 < x_k < 2**46 and 0 < a < 2**46.  This scheme
     generates 2**44 numbers before repeating.  The generated values
     are normalized to be between 0 and 1, i.e., 2**(-46) * x_k.

     This generator should produce the same results on any computer
     with at least 48 mantissa bits for `real(64)` data.

   Here is a list of currently open issues (TODOs) for this module:

   1. This module is currently restricted to generating `real(64)`,
   `imag(64)`, and `complex(128)` complex values using a single PRNG
   algorithm.  As noted above, we would like to extend this support to
   include other algorithms and primitive types over time.

   2. We plan to support general serial and parallel iterator methods
   on :class:`RandomStream`; however, providing the full suite of
   iterators is not possible with our current parallel iterator
   framework.  Specifically, if :class:`RandomStream` is a follower in
   a zippered iteration context, there is no way for it to update the
   total number of random numbers generated in a safe/sane/coordinated
   way.  We are exploring a revised leader-follower iterator framework
   that would support this idiom (and other cursor-based ones).  With
   Chapel's recent support for standalone parallel iterators, one
   could define a standalone parallel iterator for
   :class:`RandomStream`, but this effort has not yet been taken on.

   3. If no seed is provided by the user, one is chosen based on the
   current time in microseconds, allowing for some degree of
   pseudorandomness in seed selection.  The intent of
   :record:`SeedGenerators` is to provide a menu of other options
   for initializing the random stream seed, but only one option is
   implemented at present.

*/
module Random {

  /*
     Note for developers on "private" symbols:

     It is the intent that once Chapel supports a notion of 'private'
     symbols, everything prefixed with RandomPrivate will be made
     private to this module and everything prefixed with
     'RandomStreamPrivate_' will be made private to the RandomStream
     class.
  */

  /* Internal TODOs for developers:

     - should RandomStream be parameterized by the type to return?
       it seems odd that currently getNext() and getNth() return
       reals always...
  */

/*
   An instance of :record:`SeedGenerators` that provides a
   convenient means of generating seeds when the user does not wish to
   specify one manually.
*/
const SeedGenerator: SeedGenerators;

// CHPLDOC FIXME: Respect ordering of things in the file

/*
  Provides methods to help generate seeds when the user doesn't want
  to create one.  It currently only supports one such method, but the
  intention is to add more over time.

  .. note::
          Once Chapel supports static class methods,
          :const:`SeedGenerator` and :record:`SeedGenerators` should
          be combined into a single record type with static methods).

*/

record SeedGenerators {
  /*
    Generate a seed based on the current time in microseconds as
    reported by :proc:`Time.getCurrentTime`, ensuring that it
    meets the PRNG's requirements.
  */
  proc currentTime: int(64) {
    use Time;
    const seed: int(64) = getCurrentTime(unit=TimeUnits.microseconds):int(64);
    return (if seed % 2 == 0 then seed + 1 else seed) % (1:int(64) << 46);
  }
};

// CHPLDOC FEEDBACK: If easy, I'd suggest either deprecating the
// :arg <type> <name>: form or else switching the order to
// :arg <name> <type>: as it's an easy trap to simply type the same
// argument as in the function's signature.

// CHPLDOC FIXME: if the first line below is shifted one character to the
// left, it ends up being rendered like a method

// CHPLDOC FIXME: if the first line below is shifted one character to the
// right, it ends up causing a warning (promoted to error) in the .rst
// file.  It'd be preferable to have it declare the issue in terms of
// the .chpl line numbers.
//

/*
  Fill an array of `real(64)`, `imag(64)`, or `complex(128)` elements
  with pseudorandom values in parallel using a new
  :class:`RandomStream` created specifically for this call.  The
  first `arr.size` values from the stream will be assigned to the
  array's elements in row-major order for `real` and `imag` elements.
  For `complex` elements, consecutive pairs of random numbers are
  assigned to the real and imaginary components, respectively.  The
  parallelization strategy is determined by the array.

  :arg arr: The array to be filled, where T is `real(64)`, `imag(64)`, or `complex(128)`.
  :type arr: [] T

  :arg seed: The seed to use for the PRNG.  Defaults to :proc:`SeedGenerator.currentTime <SeedGenerators.currentTime>`.
  :type seed: int(64)
*/

proc fillRandom(arr: [], seed: int(64) = SeedGenerator.currentTime)
  where (arr.eltType == real || arr.eltType == imag || arr.eltType == complex) {
  var randNums = new RandomStream(seed, eltType=arr.eltType, parSafe=false);
  randNums.fillRandom(arr);
  delete randNums;
}

pragma "no doc"
proc fillRandom(arr: [], seed: int(64) = SeedGenerator.currentTime) {
  compilerError("Random.fillRandom is only defined for real(64), imag(64), and complex(128) arrays");
}

/*
  Models a stream of pseudorandom numbers.  See the module-level
  notes for :mod:`Random` for details on the PRNG used.
*/
class RandomStream {
  /*
    Specifies the type of value generated by the RandomStream.
    Currently, only `real(64)`, `imag(64)`, and `complex(128)` are
    supported.
  */
  type eltType = real(64);

  /*
    Indicates whether or not the RandomStream needs to be
    parallel-safe by default.  If multiple tasks interact with it in
    an uncoordinated fashion, this must be set to `true`.  If it will
    only be called from a single task, or if only one task will call
    into it at a time, setting to `false` will reduce overhead related
    to ensuring mutual exclusion.
  */
  param parSafe: bool = true;

  /*
    The seed value for the PRNG.  It must be an odd integer in the
    range (1, 2**46).
  */
  const seed: int(64);

  /*
    Constructs a new stream of random numbers using the specified seed
    and parallel safety.  Ensures that the seed value meets the PRNG's
    constraints.

    :arg seed: The seed to use for the PRNG.  Defaults to :proc:`SeedGenerator.currentTime <SeedGenerators.currentTime>`..
    :type seed: int(64)

    :arg parSafe: The parallel safety setting.  Defaults to `true`.
    :type parSafe: bool

    :arg eltType: The element type to be generated.  Defaults to `real(64)`.
    :type eltType: type
  */
  proc RandomStream(seed: int(64) = SeedGenerator.currentTime,
                    param parSafe: bool = true,
                    type eltType = real(64)) {
    if seed % 2 == 0 || seed < 1 || seed > 1:int(64)<<46 then
      halt("RandomStream seed must be an odd integer between 0 and 2**46");
    this.seed = seed;
    RandomStreamPrivate_cursor = seed;
    RandomStreamPrivate_count = 1;
  }

  pragma "no doc"
  proc RandomStreamPrivate_getNext_noLock() {
    if (eltType == complex) {
      RandomStreamPrivate_count += 2;
    } else {
      RandomStreamPrivate_count += 1;
    }
    return RandomPrivate_randlc(eltType, RandomStreamPrivate_cursor);
  }

  pragma "no doc"
  proc RandomStreamPrivate_skipToNth_noLock(in n: integral) {
    if eltType == complex then n = n*2 - 1;
    RandomStreamPrivate_count = n;
    RandomStreamPrivate_cursor = RandomPrivate_randlc_skipto(seed, n);
  }

  /*
    Returns the next value in the random stream.

    :returns: The next value in the random stream as type :type:`eltType`.
   */
  proc getNext(): eltType {
    if parSafe then
      RandomStreamPrivate_lock$ = true;
    const result = RandomStreamPrivate_getNext_noLock();
    if parSafe then
      RandomStreamPrivate_lock$;
    return result;
  }

  /*
    Advances/rewinds the stream to the `n`-th value in the sequence.

    :arg n: The position in the stream to skip to.  Must be non-negative.
    :type n: integral
   */

  proc skipToNth(n: integral) {
    if n <= 0 then
      halt("RandomStream.skipToNth(n) called with non-positive 'n' value", n);
    if parSafe then
      RandomStreamPrivate_lock$ = true;
    RandomStreamPrivate_skipToNth_noLock(n);
    if parSafe then
      RandomStreamPrivate_lock$;
  }

  /*
    Advance/rewind the stream to the `n`-th value and return it
    (advancing the stream by one).  This is equivalent to
    :proc:`skipToNth()` followed by :proc:`getNext()`.

    :arg n: The position in the stream to skip to.  Must be non-negative.
    :type n: integral

    :returns: The `n`-th value in the random stream as type :type:`eltType`.
  */

  proc getNth(n: integral): eltType {
    if (n <= 0) then
      halt("RandomStream.getNth(n) called with non-positive 'n' value", n);
    if parSafe then
      RandomStreamPrivate_lock$ = true;
    RandomStreamPrivate_skipToNth_noLock(n);
    const result = RandomStreamPrivate_getNext_noLock();
    if parSafe then
      RandomStreamPrivate_lock$;
    return result;
  }

  /*
    Fill the argument array with pseudorandom values.  This method is
    identical to the standalone :proc:`fillRandom` procedure,
    except that it consumes random values from the
    :class:`RandomStream` object on which it's invoked rather
    than creating a new stream for the purpose of the call.

    :arg arr: The array to be filled
    :type arr: [] :type:`eltType`
  */
  proc fillRandom(arr: [] eltType) {
    forall (x, r) in zip(arr, iterate(arr.domain, arr.eltType)) do
      x = r;
  }

  pragma "no doc"
  proc fillRandom(arr: []) {
    compilerError("RandomStream(eltType=", typeToString(eltType), ") can only be used to fill arrays of ", typeToString(eltType));
  }

  pragma "no doc"
  proc iterate(D: domain, type resultType=real) {
    if parSafe then
      RandomStreamPrivate_lock$ = true;
    const start = RandomStreamPrivate_count;
    RandomStreamPrivate_count += D.numIndices.safeCast(int(64));
    RandomStreamPrivate_skipToNth_noLock(RandomStreamPrivate_count);
    if parSafe then
      RandomStreamPrivate_lock$;
    return RandomPrivate_iterate(resultType, D, seed, start);
  }

  pragma "no doc"
  proc writeThis(f: Writer) {
    f <~> "RandomStream(eltType=";
    f <~> typeToString(eltType);
    f <~> ", parSafe=";
    f <~> parSafe;
    f <~> ", seed=";
    f <~> seed;
    f <~> ")";
  }

  ///////////////////////////////////////////////////////////// CLASS PRIVATE //
  //
  // It is the intent that once Chapel supports the notion of
  // 'private', everything in this class declared below this line will
  // be made private to this class.
  //

  pragma "no doc"
  var RandomStreamPrivate_lock$: sync bool;
  pragma "no doc"
  var RandomStreamPrivate_cursor: real = seed;
  pragma "no doc"
  var RandomStreamPrivate_count: int(64) = 1;
}

////////////////////////////////////////////////////////////// MODULE PRIVATE //
//
// It is the intent that once Chapel supports the notion of 'private',
// everything declared below this line will be made private to this
// module.
//

//
// NPB-defined constants for linear congruential generator
//
pragma "no doc"
const RandomPrivate_r23   = 0.5**23,
      RandomPrivate_t23   = 2.0**23,
      RandomPrivate_r46   = 0.5**46,
      RandomPrivate_t46   = 2.0**46,
      RandomPrivate_arand = 1220703125.0; // TODO: Is arand something that a
                                          // user might want to set on a
                                          // case-by-case basis?

//
// NPB-defined randlc routine
//
pragma "no doc"
proc RandomPrivate_randlc(inout x: real, a: real = RandomPrivate_arand) {
  var t1 = RandomPrivate_r23 * a;
  const a1 = floor(t1),
    a2 = a - RandomPrivate_t23 * a1;
  t1 = RandomPrivate_r23 * x;
  const x1 = floor(t1),
    x2 = x - RandomPrivate_t23 * x1;
  t1 = a1 * x2 + a2 * x1;
  const t2 = floor(RandomPrivate_r23 * t1),
    z  = t1 - RandomPrivate_t23 * t2,
    t3 = RandomPrivate_t23 * z + a2 * x2,
    t4 = floor(RandomPrivate_r46 * t3),
    x3 = t3 - RandomPrivate_t46 * t4;
  x = x3;
  return RandomPrivate_r46 * x3;
}

// Wrapper that takes a result type (two calls for complex types)
pragma "no doc"
proc RandomPrivate_randlc(type resultType, inout x: real) {
  if resultType == complex then
    return (RandomPrivate_randlc(x), RandomPrivate_randlc(x)):complex;
  else
    if resultType == imag then
      //
      // BLC: I thought that casting real to imag did this automatically?
      //
      return _r2i(RandomPrivate_randlc(x));
    else
      return RandomPrivate_randlc(x);
 }

//
// Return a value for the cursor so that the next call to randlc will
// return the same value as the nth call to randlc
//
pragma "no doc"
proc RandomPrivate_randlc_skipto(seed: int(64), in n: integral): real {
  var cursor = seed:real;
  n -= 1;
  var t = RandomPrivate_arand;
  RandomPrivate_arand;
  while (n != 0) {
    const i = n / 2;
    if (2 * i != n) then
      RandomPrivate_randlc(cursor, t);
    if i == 0 then
      break;
    else
      n = i;
    RandomPrivate_randlc(t, t);
    n = i;
  }
  return cursor;
}

//
// iterate over outer ranges in tuple of ranges
//
pragma "no doc"
iter RandomPrivate_outer(ranges, param dim: int = 1) {
  if dim + 1 == ranges.size {
    for i in ranges(dim) do
      yield (i,);
  } else if dim + 1 < ranges.size {
    for i in ranges(dim) do
      for j in RandomPrivate_outer(ranges, dim+1) do
        yield (i, (...j));
  } else {
    yield 0; // 1D case is a noop
  }
}

//
// RandomStream iterator implementation
//
pragma "no doc"
iter RandomPrivate_iterate(type resultType, D: domain, seed: int(64),
                          start: int(64)) {
  var cursor = RandomPrivate_randlc_skipto(seed, start);
  for i in D do
    yield RandomPrivate_randlc(resultType, cursor);
}

pragma "no doc"
iter RandomPrivate_iterate(type resultType, D: domain, seed: int(64),
                          start: int(64), param tag: iterKind)
      where tag == iterKind.leader {
  for block in D._value.these(tag=iterKind.leader) do
    yield block;
}

pragma "no doc"
iter RandomPrivate_iterate(type resultType, D: domain, seed: int(64),
                          start: int(64), param tag: iterKind, followThis)
      where tag == iterKind.follower {
  param multiplier = if resultType == complex then 2 else 1;
  const ZD = computeZeroBasedDomain(D);
  const innerRange = followThis(ZD.rank);
  var cursor: real;
  for outer in RandomPrivate_outer(followThis) {
    var myStart = start;
    if ZD.rank > 1 then
      myStart += multiplier * ZD.indexOrder(((...outer), innerRange.low)).safeCast(int(64));
    else
      myStart += multiplier * ZD.indexOrder(innerRange.low).safeCast(int(64));
    if !innerRange.stridable {
      cursor = RandomPrivate_randlc_skipto(seed, myStart);
      for i in innerRange do
        yield RandomPrivate_randlc(resultType, cursor);
    } else {
      myStart -= innerRange.low.safeCast(int(64));
      for i in innerRange {
        cursor = RandomPrivate_randlc_skipto(seed, myStart + i.safeCast(int(64)) * multiplier);
        yield RandomPrivate_randlc(resultType, cursor);
      }
    }
  }
}

} // close module Random


Cache text.size to improve performance for large strings

extern type qio_regexp_t;

extern record qio_regexp_options_t {
  var utf8:bool;
  var posix:bool;
  var literal:bool;
  var nocapture:bool;
  // These ones can be set inside the regexp
  var ignorecase:bool; // (?i)
  var multiline:bool; // (?m)
  var dotnl:bool; // (?s)
  var nongreedy:bool; // (?U)
}

extern proc qio_regexp_null():qio_regexp_t;
extern proc qio_regexp_init_default_options(ref options:qio_regexp_options_t);
extern proc qio_regexp_create_compile(str:c_string, strlen:int(64), ref options:qio_regexp_options_t, ref compiled:qio_regexp_t);
extern proc qio_regexp_create_compile_flags(str:c_string, strlen:int(64), flags:c_string, flagslen:int(64), isUtf8:bool, ref compiled:qio_regexp_t);
extern proc qio_regexp_create_compile_flags_2(str:c_void_ptr, strlen:int(64), flags:c_void_ptr, flagslen:int(64), isUtf8:bool, ref compiled:qio_regexp_t);
extern proc qio_regexp_retain(ref compiled:qio_regexp_t);
extern proc qio_regexp_release(ref compiled:qio_regexp_t);

extern proc qio_regexp_get_options(ref regexp:qio_regexp_t, ref options: qio_regexp_options_t);
extern proc qio_regexp_get_pattern(ref regexp:qio_regexp_t, ref pattern: c_string);
extern proc qio_regexp_get_ncaptures(ref regexp:qio_regexp_t):int(64);
extern proc qio_regexp_ok(ref regexp:qio_regexp_t):bool;
extern proc qio_regexp_error(ref regexp:qio_regexp_t):c_string;

extern const QIO_REGEXP_ANCHOR_UNANCHORED:c_int;
extern const QIO_REGEXP_ANCHOR_START:c_int;
extern const QIO_REGEXP_ANCHOR_BOTH:c_int;

extern record qio_regexp_string_piece_t {
  var offset:int(64); // counting from 0, -1 means "NULL"
  var len:int(64);
}

extern proc qio_regexp_string_piece_isnull(ref sp:qio_regexp_string_piece_t):bool;

extern proc qio_regexp_match(ref re:qio_regexp_t, text:c_string, textlen:int(64), startpos:int(64), endpos:int(64), anchor:c_int, submatch:_ddata(qio_regexp_string_piece_t), nsubmatch:int(64)):bool;
extern proc qio_regexp_replace(ref re:qio_regexp_t, repl:c_string, repllen:int(64), text:c_string, textlen:int(64), startpos:int(64), endpos:int(64), global:bool, ref replaced:c_string, ref replaced_len:int(64)):int(64);

// These two could be folded together if we had a way
// to check if a default argument was supplied
// (or any way to use 'nil' in pass-by-ref)
/** Compile a regular expression. This routine will halt if compilation failed
 */
proc compile(pattern: string, utf8=true, posix=false, literal=false, nocapture=false, /*i*/ ignorecase=false, /*m*/ multiline=false, /*s*/ dotnl=false, /*U*/ nongreedy=false):regexp {
  var opts:qio_regexp_options_t;
  qio_regexp_init_default_options(opts);
  opts.utf8 = utf8;
  opts.posix = posix;
  opts.literal = literal;
  opts.nocapture = nocapture;
  opts.ignorecase = ignorecase;
  opts.multiline = multiline;
  opts.dotnl = dotnl;
  opts.nongreedy = nongreedy;

  var ret:regexp;
  qio_regexp_create_compile(pattern.c_str(), pattern.size, opts, ret._regexp);
  if ! qio_regexp_ok(ret._regexp) {
    var err_str = qio_regexp_error(ret._regexp);
    __primitive("chpl_error", "Error " + err_str + " when compiling regexp '" + pattern + "'");
  }
  return ret;
}

/** Compile a regular expression and retrieve an error code if compilation
    failed.
 */
proc compile(pattern: string, out error:syserr, utf8=true, posix=false, literal=false, nocapture=false, /*i*/ ignorecase=false, /*m*/ multiline=false, /*s*/ dotnl=false, /*U*/ nongreedy=false):regexp {
  var opts:qio_regexp_options_t;
  qio_regexp_init_default_options(opts);
  opts.utf8 = utf8;
  opts.posix = posix;
  opts.literal = literal;
  opts.nocapture = nocapture;
  opts.ignorecase = ignorecase;
  opts.multiline = multiline;
  opts.dotnl = dotnl;
  opts.nongreedy = nongreedy;

  var ret:regexp;
  qio_regexp_create_compile(pattern, pattern.size, opts, ret._regexp);

  if qio_regexp_ok(ret._regexp) {
    error = ENOERR;
  } else {
    error = qio_format_error_bad_regexp();
  }

  return ret;
}

/** Regular expression search routines also can work with
    a stringPart, which refers to a substring within another
    string.
 */
record stringPart {
  var offset:int;
  var length:int;
  var from:string;
}

/** The reMatch record records a regular expression search match
    or a capture group.
    Regular expression search routines normally return one of these.
    Also, this type can be passed as a capture group argument.
    Lastly, something of type reMatch can be checked for a match
    in a simple if statement, as in:
      var m:reMatch = ....;
      if m then do_something_if_matched();
      if !m then do_something_if_not_matched();

 */
record reMatch {
  var matched:bool;
  var offset:int; // 0-based, -1 if matched==false
  var length:int; // 0 if matched==false
}

proc _to_reMatch(ref p:qio_regexp_string_piece_t):reMatch {
  if qio_regexp_string_piece_isnull(p) {
    return new reMatch(false, -1, 0);
  } else {
    return new reMatch(true, p.offset, p.len);
  }
}

inline proc !(m: reMatch) return !m.matched;
inline proc _cond_test(m: reMatch) return m.matched;

/** Use someString.substring(anReMatch) to extract the part
    of a string matching a regular expression or capture group.
 */
proc string.substring(m:reMatch) {
  if m.matched then return this.substring(m.offset+1..#m.length);
  else return "";
}

/** This class represents a compiled regular expression. Regular expressions
    are cached on a per-thread basis and are reference counted.
    To create a compiled regular expression, use the compile function.
  */
pragma "ignore noinit"
record regexp {
  var home: locale = here;
  var _regexp:qio_regexp_t = qio_regexp_null();

  proc ok:bool {
    return qio_regexp_ok(_regexp);
  }
  proc error():string {
    return qio_regexp_error(_regexp);
  }

  // note - more = overloads are below.
  proc ~regexp() {
    qio_regexp_release(_regexp);
    _regexp = qio_regexp_null();
  }

  proc _handle_captures(text: string, matches:_ddata(qio_regexp_string_piece_t), nmatches:int, ref captures) {
    assert(nmatches >= captures.size);
    for param i in 1..captures.size {
      var m = _to_reMatch(matches[i]);
      if captures[i].type == reMatch {
        captures[i] = m;
      } else {
        if m.matched {
          captures[i] = text.substring(m):captures[i].type;
        } else {
          var empty:captures[i].type;
          captures[i] = empty;
        }
      }
    }
  }

  // Note - we would only need one version of these routines
  // if we had args ...?k supporting 0 args, or if tuples support zero length
  /** Search within the passed text (which must be a string or a stringPart)
      for the first match at any offset to this regular expression. Returns a
      match object. This routine will try matching the regular expression at
      different offsets until a match is found. If you want to only match at
      the beginning of the pattern, you can start your pattern with ^ or
      use regexp.match. The capture group arguments here can be strings
      or any type that can reasonably cast from string. If a capture group
      was not matched, the corresponding argument will get the default value
      for its type.
    */
  proc search(text: ?t, ref captures ...?k):reMatch
    where t == string || t == stringPart
  {
    var ret:reMatch;
    on this.home {
      var pos:int;
      var endpos:int;
      if t == stringPart {
        pos = text.offset;
        endpos = pos + text.length;
      } else {
        pos = 0;
        endpos = text.size;
      }
      var matches:_ddata(qio_regexp_string_piece_t);
      var nmatches = 1 + captures.size;
      matches = _ddata_allocate(qio_regexp_string_piece_t, nmatches);
      var got:bool;
      if t == stringPart {
       got = qio_regexp_match(_regexp, text.from.c_str(), text.from.size, pos, endpos, QIO_REGEXP_ANCHOR_UNANCHORED, matches, nmatches);
      } else {
       got = qio_regexp_match(_regexp, text.c_str(), text.size, pos, endpos, QIO_REGEXP_ANCHOR_UNANCHORED, matches, nmatches);
      }
      // Now try to coerce the read strings into the captures.
      _handle_captures(text, matches, nmatches, captures);
      // Now return where we matched.
      ret = new reMatch(got, matches[0].offset, matches[0].len);
      _ddata_free(matches);
    }
    return ret;
  }
  /** Search for the first match at any offset without returning any capture
      groups.
    */
  proc search(text: ?t):reMatch
    where t == string || t == stringPart
  {
    var ret:reMatch;
    on this.home {
      var pos:int;
      var endpos:int;
      if t == stringPart {
        pos = text.offset;
        endpos = pos + text.length;
      } else {
        pos = 0;
        endpos = text.size;
      }
      var matches:_ddata(qio_regexp_string_piece_t);
      var nmatches = 1;
      matches = _ddata_allocate(qio_regexp_string_piece_t, nmatches);
      var got:bool;
      if t == stringPart {
       got = qio_regexp_match(_regexp, text.from.c_str(), text.from.size, pos, endpos, QIO_REGEXP_ANCHOR_UNANCHORED, matches, nmatches);
      } else {
       got = qio_regexp_match(_regexp, text.c_str(), text.size, pos, endpos, QIO_REGEXP_ANCHOR_UNANCHORED, matches, nmatches);
      }
      // Now return where we matched.
      ret = new reMatch(got, matches[0].offset, matches[0].len);
      _ddata_free(matches);
    }
    return ret;
  }

  /** Check for a match to this regular expression at the start of the
      passed text. As with regexp.search, the capture group arguments
      here can be strings or any type that can reasonably cast from string. If
      a capture group was not matched, the corresponding argument will get the
      default value for its type.

      You can do
       if myregexp.match("some string") {
         ... do something if matched;
       }
   */
  proc match(text: ?t, ref captures ...?k):reMatch
    where t == string || t == stringPart
  {
    var ret:reMatch;
    on this.home {
      var pos:int;
      var endpos:int;
      if t == stringPart {
        pos = text.offset;
        endpos = pos + text.length;
      } else {
        pos = 0;
        endpos = text.size;
      }
      var matches:_ddata(qio_regexp_string_piece_t);
      var nmatches = 1 + captures.size;
      matches = _ddata_allocate(qio_regexp_string_piece_t, nmatches);
      var got:bool;
      if t == stringPart {
        got = qio_regexp_match(_regexp, text.from.c_str(), text.from.size, pos, endpos, QIO_REGEXP_ANCHOR_START, matches, nmatches);
      } else {
        got = qio_regexp_match(_regexp, text.c_str(), text.size, pos, endpos, QIO_REGEXP_ANCHOR_START, matches, nmatches);
      }
      // Now try to coerce the read strings into the captures.
      _handle_captures(text, matches, nmatches, captures);
      // Now return where we matched.
      ret = new reMatch(got, matches[0].offset, matches[0].len);
      _ddata_free(matches);
    }
    return ret;
  }
  proc match(text: ?t):reMatch
    where t == string || t == stringPart
  {
    var ret:reMatch;
    on this.home {
      var pos:int;
      var endpos:int;
      if t == stringPart {
        pos = text.offset;
        endpos = pos + text.length;
      } else {
        pos = 0;
        endpos = text.size;
      }
      var matches:_ddata(qio_regexp_string_piece_t);
      var nmatches = 1;
      matches = _ddata_allocate(qio_regexp_string_piece_t, nmatches);
      var got:bool;
      if t == stringPart {
       got = qio_regexp_match(_regexp, text.from.c_str(), text.from.size, pos, endpos, QIO_REGEXP_ANCHOR_START, matches, nmatches);
      } else {
       got = qio_regexp_match(_regexp, text.c_str(), text.size, pos, endpos, QIO_REGEXP_ANCHOR_START, matches, nmatches);
      }
      // Now return where we matched.
      ret = new reMatch(got, matches[0].offset, matches[0].len);
      _ddata_free(matches);
    }
    return ret;
  }

  /** Split the text by occurrences of this regular expression.
     If capturing parentheses are used in pattern, then the text of all
     groups in the pattern are also returned as part of the resulting array.
     If maxsplit is nonzero, at most maxsplit splits occur, and the
     remaining text is returned as the last element.
   */
  iter split(text: ?t, maxsplit: int = 0)
    where t == string || t == stringPart
  {
    var matches:_ddata(qio_regexp_string_piece_t);
    var ncaptures = qio_regexp_get_ncaptures(_regexp);
    var nmatches = 1 + ncaptures;
    var pos:int;
    var endpos:int;

    on this.home {
      matches = _ddata_allocate(qio_regexp_string_piece_t, nmatches);
    }

    if t == stringPart {
      pos = text.offset;
      endpos = pos + text.length;
    } else {
      pos = 0;
      endpos = text.size;
    }
    var splits = 0;
    var maxsplits = maxsplit;
    if maxsplit == 0 then maxsplits = max(int);

    while true {
      var splitstart = 0;
      var splitend = 0;
      var got:bool;
      on this.home {
        got = qio_regexp_match(_regexp, text.c_str(), text.size, pos, endpos, QIO_REGEXP_ANCHOR_UNANCHORED, matches, nmatches);
      }

      splits += 1;
      if got && splits <= maxsplits {
        splitstart = matches[0].offset;
        splitend = matches[0].offset + matches[0].len;
      } else {
        splitstart = endpos;
        splitend = endpos;
      }

      if pos < endpos {
        // Yield splitted value
        yield text.substring(pos+1..splitstart);
      } else {
        yield "";
      }

      if got {
        // Yield capture groups
        for i in 1..ncaptures {
          yield text.substring(new reMatch(
                !qio_regexp_string_piece_isnull(matches[i]),
                matches[i].offset,
                matches[i].len));
        }
      }

      // Advance to splitend.
      pos = splitend;

      if splits > maxsplits || !got then break;
    }
    on this.home {
      _ddata_free(matches);
    }
  }
  /* Enumerates matches in the string as well as capture groups.
     Returns tuples of reMatch objects, the 1st is always
      the match for the whole pattern and the rest are the capture groups.
   */
  iter matches(text: ?t, param captures=0, maxmatches: int = max(int))
    where t == string || t == stringPart
  {
    var matches:_ddata(qio_regexp_string_piece_t);
    var nmatches = 1 + captures;
    var pos:int;
    var endpos:int;
    var textSize:int;
    on this.home {
      matches = _ddata_allocate(qio_regexp_string_piece_t, nmatches);
    }
    if t == stringPart {
      pos = text.offset;
      endpos = pos + text.length;
      textSize = text.from.size;
    } else {
      pos = 0;
      endpos = text.size;
      textSize = text.size;
    }
    var nfound = 0;
    var cur = pos;
    while nfound < maxmatches && cur < endpos {
      var got:bool;
      on this.home {
        got = qio_regexp_match(_regexp, text.c_str(), textSize, cur, endpos, QIO_REGEXP_ANCHOR_UNANCHORED, matches, nmatches);
      }
      if !got then break;
      param nret = captures+1;
      var ret:nret*reMatch;
      for i in 0..captures {
        ret[i+1] = new reMatch(got, matches[i].offset, matches[i].len);
      }
      yield ret;
      cur = matches[0].offset + matches[0].len;
    }
    on this.home {
      _ddata_free(matches);
    }
  }
  proc subn(repl:string, text: ?t, global = true ):(string, int)
    where t == string || t == stringPart
  {
    var pos:int;
    var endpos:int;
    if t == stringPart {
      pos = text.offset;
      endpos = pos + text.length;
    } else {
      pos = 0;
      endpos = text.size;
    }
    var replaced:c_string; // FIX ME: leak c_string
    var nreplaced:int;
    var replaced_len:int(64);
    if t == stringPart {
      nreplaced = qio_regexp_replace(_regexp, repl.c_str(), repl.size, text.from.c_str(), text.from.size, pos, endpos, global, replaced, replaced_len);
    } else {
      nreplaced = qio_regexp_replace(_regexp, repl.c_str(), repl.size, text.c_str(), text.size, pos, endpos, global, replaced, replaced_len);
    }
    return (toString(replaced), nreplaced);
  }
  proc sub(repl:string, text: ?t, global = true )
    where t == string || t == stringPart
  {
    var (str, count) = subn(repl, text, global);
    return str;
  }
  proc writeThis(f: Writer) {
    var pattern:c_string;
    on this.home {
      qio_regexp_get_pattern(this._regexp, pattern);
    }
    // Note -- this is wrong because we didn't quote
    // and there's no way to get the flags
    f.write("new regexp(\"", pattern, "\")");
  }
  proc readThis(f: Reader) {
    var pattern:c_string;
    // Note -- this is wrong because we didn't quote
    // and there's no way to get the flags
    var litOne = new ioLiteral("new regexp(\"");
    var litTwo = new ioLiteral("\")");
    if(f.read(litOne, pattern, litTwo)) {
      on this.home {
        var opts:qio_regexp_options_t;
        qio_regexp_init_default_options(opts);
        qio_regexp_create_compile(pattern, pattern.size, opts, this._regexp);
      }
    }
  }
}

proc =(ref ret:regexp, x:regexp)
{
  // retain -- release
  if x.home == ret.home {
    on x.home {
      qio_regexp_retain(x._regexp);
      qio_regexp_release(ret._regexp);
    }
    ret._regexp = x._regexp;
  } else {
    var pattern:c_string;
    var options:qio_regexp_options_t;

    on ret.home {
      qio_regexp_release(ret._regexp);
    }
    on x.home {
      qio_regexp_get_pattern(x._regexp, pattern);
      qio_regexp_get_options(x._regexp, options);
    }

    qio_regexp_create_compile(pattern, pattern.size, options, ret._regexp);
  }
}

// TODO -- shouldn't have to write this this way!
pragma "init copy fn"
proc chpl__initCopy(x: regexp) {
  on x.home {
    qio_regexp_retain(x._regexp);
  }
  return x;
}

// Cast regexp to string.
inline proc _cast(type t, x: regexp) where t == string {
  var pattern:c_string;
  on x.home {
    qio_regexp_get_pattern(x._regexp, pattern);
  }
  // FIX ME: leak c_string
  return toString(pattern);
}
// Cast string to regexp
inline proc _cast(type t, x: string) where t == regexp {
  return compile(x);
}

// string methods for searching
proc string.search(needle: string, ignorecase=false):reMatch
{
  // Create a regexp matching the literal for needle
  var re = compile(needle, literal=true, nocapture=true, ignorecase=ignorecase);
  return re.search(this);
}
proc string.search(needle: regexp):reMatch
{
  return needle.search(this);
}
proc string.search(needle: regexp, ref captures ...?k):reMatch
{
  return needle.search(this, (...captures));
}
proc string.match(pattern: regexp):reMatch
{
  return pattern.match(this);
}
proc string.match(pattern: regexp, ref captures ...?k):reMatch
{
  return pattern.match(this, (...captures));
}

iter string.split(pattern: regexp, maxsplit: int = 0)
{
  for v in pattern.split(this, maxsplit) {
    yield v;
  }
}

iter string.matches(pattern:regexp, param captures=0, maxmatches:int=max(int))
{
  for v in pattern.matches(this, captures, maxmatches) {
    yield v;
  }
}

extern type qio_regexp_t;

extern record qio_regexp_options_t {
  var utf8:bool;
  var posix:bool;
  var literal:bool;
  var nocapture:bool;
  // These ones can be set inside the regexp
  var ignorecase:bool; // (?i)
  var multiline:bool; // (?m)
  var dotnl:bool; // (?s)
  var nongreedy:bool; // (?U)
}

extern proc qio_regexp_null():qio_regexp_t;
extern proc qio_regexp_init_default_options(ref options:qio_regexp_options_t);
extern proc qio_regexp_create_compile(str:c_string, strlen:int(64), ref options:qio_regexp_options_t, ref compiled:qio_regexp_t);
extern proc qio_regexp_create_compile_flags(str:c_string, strlen:int(64), flags:c_string, flagslen:int(64), isUtf8:bool, ref compiled:qio_regexp_t);
extern proc qio_regexp_create_compile_flags_2(str:c_void_ptr, strlen:int(64), flags:c_void_ptr, flagslen:int(64), isUtf8:bool, ref compiled:qio_regexp_t);
extern proc qio_regexp_retain(ref compiled:qio_regexp_t);
extern proc qio_regexp_release(ref compiled:qio_regexp_t);

extern proc qio_regexp_get_options(ref regexp:qio_regexp_t, ref options: qio_regexp_options_t);
extern proc qio_regexp_get_pattern(ref regexp:qio_regexp_t, ref pattern: c_string);
extern proc qio_regexp_get_ncaptures(ref regexp:qio_regexp_t):int(64);
extern proc qio_regexp_ok(ref regexp:qio_regexp_t):bool;
extern proc qio_regexp_error(ref regexp:qio_regexp_t):c_string;

extern const QIO_REGEXP_ANCHOR_UNANCHORED:c_int;
extern const QIO_REGEXP_ANCHOR_START:c_int;
extern const QIO_REGEXP_ANCHOR_BOTH:c_int;

extern record qio_regexp_string_piece_t {
  var offset:int(64); // counting from 0, -1 means "NULL"
  var len:int(64);
}

extern proc qio_regexp_string_piece_isnull(ref sp:qio_regexp_string_piece_t):bool;

extern proc qio_regexp_match(ref re:qio_regexp_t, text:c_string, textlen:int(64), startpos:int(64), endpos:int(64), anchor:c_int, submatch:_ddata(qio_regexp_string_piece_t), nsubmatch:int(64)):bool;
extern proc qio_regexp_replace(ref re:qio_regexp_t, repl:c_string, repllen:int(64), text:c_string, textlen:int(64), startpos:int(64), endpos:int(64), global:bool, ref replaced:c_string, ref replaced_len:int(64)):int(64);

// These two could be folded together if we had a way
// to check if a default argument was supplied
// (or any way to use 'nil' in pass-by-ref)
/** Compile a regular expression. This routine will halt if compilation failed
 */
proc compile(pattern: string, utf8=true, posix=false, literal=false, nocapture=false, /*i*/ ignorecase=false, /*m*/ multiline=false, /*s*/ dotnl=false, /*U*/ nongreedy=false):regexp {
  var opts:qio_regexp_options_t;
  qio_regexp_init_default_options(opts);
  opts.utf8 = utf8;
  opts.posix = posix;
  opts.literal = literal;
  opts.nocapture = nocapture;
  opts.ignorecase = ignorecase;
  opts.multiline = multiline;
  opts.dotnl = dotnl;
  opts.nongreedy = nongreedy;

  var ret:regexp;
  qio_regexp_create_compile(pattern.c_str(), pattern.size, opts, ret._regexp);
  if ! qio_regexp_ok(ret._regexp) {
    var err_str = qio_regexp_error(ret._regexp);
    __primitive("chpl_error", "Error " + err_str + " when compiling regexp '" + pattern + "'");
  }
  return ret;
}

/** Compile a regular expression and retrieve an error code if compilation
    failed.
 */
proc compile(pattern: string, out error:syserr, utf8=true, posix=false, literal=false, nocapture=false, /*i*/ ignorecase=false, /*m*/ multiline=false, /*s*/ dotnl=false, /*U*/ nongreedy=false):regexp {
  var opts:qio_regexp_options_t;
  qio_regexp_init_default_options(opts);
  opts.utf8 = utf8;
  opts.posix = posix;
  opts.literal = literal;
  opts.nocapture = nocapture;
  opts.ignorecase = ignorecase;
  opts.multiline = multiline;
  opts.dotnl = dotnl;
  opts.nongreedy = nongreedy;

  var ret:regexp;
  qio_regexp_create_compile(pattern, pattern.size, opts, ret._regexp);

  if qio_regexp_ok(ret._regexp) {
    error = ENOERR;
  } else {
    error = qio_format_error_bad_regexp();
  }

  return ret;
}

/** Regular expression search routines also can work with
    a stringPart, which refers to a substring within another
    string.
 */
record stringPart {
  var offset:int;
  var length:int;
  var from:string;
}

/** The reMatch record records a regular expression search match
    or a capture group.
    Regular expression search routines normally return one of these.
    Also, this type can be passed as a capture group argument.
    Lastly, something of type reMatch can be checked for a match
    in a simple if statement, as in:
      var m:reMatch = ....;
      if m then do_something_if_matched();
      if !m then do_something_if_not_matched();

 */
record reMatch {
  var matched:bool;
  var offset:int; // 0-based, -1 if matched==false
  var length:int; // 0 if matched==false
}

proc _to_reMatch(ref p:qio_regexp_string_piece_t):reMatch {
  if qio_regexp_string_piece_isnull(p) {
    return new reMatch(false, -1, 0);
  } else {
    return new reMatch(true, p.offset, p.len);
  }
}

inline proc !(m: reMatch) return !m.matched;
inline proc _cond_test(m: reMatch) return m.matched;

/** Use someString.substring(anReMatch) to extract the part
    of a string matching a regular expression or capture group.
 */
proc string.substring(m:reMatch) {
  if m.matched then return this.substring(m.offset+1..#m.length);
  else return "";
}

/** This class represents a compiled regular expression. Regular expressions
    are cached on a per-thread basis and are reference counted.
    To create a compiled regular expression, use the compile function.
  */
pragma "ignore noinit"
record regexp {
  var home: locale = here;
  var _regexp:qio_regexp_t = qio_regexp_null();

  proc ok:bool {
    return qio_regexp_ok(_regexp);
  }
  proc error():string {
    return qio_regexp_error(_regexp);
  }

  // note - more = overloads are below.
  proc ~regexp() {
    qio_regexp_release(_regexp);
    _regexp = qio_regexp_null();
  }

  proc _handle_captures(text: string, matches:_ddata(qio_regexp_string_piece_t), nmatches:int, ref captures) {
    assert(nmatches >= captures.size);
    for param i in 1..captures.size {
      var m = _to_reMatch(matches[i]);
      if captures[i].type == reMatch {
        captures[i] = m;
      } else {
        if m.matched {
          captures[i] = text.substring(m):captures[i].type;
        } else {
          var empty:captures[i].type;
          captures[i] = empty;
        }
      }
    }
  }

  // Note - we would only need one version of these routines
  // if we had args ...?k supporting 0 args, or if tuples support zero length
  /** Search within the passed text (which must be a string or a stringPart)
      for the first match at any offset to this regular expression. Returns a
      match object. This routine will try matching the regular expression at
      different offsets until a match is found. If you want to only match at
      the beginning of the pattern, you can start your pattern with ^ or
      use regexp.match. The capture group arguments here can be strings
      or any type that can reasonably cast from string. If a capture group
      was not matched, the corresponding argument will get the default value
      for its type.
    */
  proc search(text: ?t, ref captures ...?k):reMatch
    where t == string || t == stringPart
  {
    var ret:reMatch;
    on this.home {
      var pos:int;
      var endpos:int;
      if t == stringPart {
        pos = text.offset;
        endpos = pos + text.length;
      } else {
        pos = 0;
        endpos = text.size;
      }
      var matches:_ddata(qio_regexp_string_piece_t);
      var nmatches = 1 + captures.size;
      matches = _ddata_allocate(qio_regexp_string_piece_t, nmatches);
      var got:bool;
      if t == stringPart {
       got = qio_regexp_match(_regexp, text.from.c_str(), text.from.size, pos, endpos, QIO_REGEXP_ANCHOR_UNANCHORED, matches, nmatches);
      } else {
       got = qio_regexp_match(_regexp, text.c_str(), text.size, pos, endpos, QIO_REGEXP_ANCHOR_UNANCHORED, matches, nmatches);
      }
      // Now try to coerce the read strings into the captures.
      _handle_captures(text, matches, nmatches, captures);
      // Now return where we matched.
      ret = new reMatch(got, matches[0].offset, matches[0].len);
      _ddata_free(matches);
    }
    return ret;
  }
  /** Search for the first match at any offset without returning any capture
      groups.
    */
  proc search(text: ?t):reMatch
    where t == string || t == stringPart
  {
    var ret:reMatch;
    on this.home {
      var pos:int;
      var endpos:int;
      if t == stringPart {
        pos = text.offset;
        endpos = pos + text.length;
      } else {
        pos = 0;
        endpos = text.size;
      }
      var matches:_ddata(qio_regexp_string_piece_t);
      var nmatches = 1;
      matches = _ddata_allocate(qio_regexp_string_piece_t, nmatches);
      var got:bool;
      if t == stringPart {
       got = qio_regexp_match(_regexp, text.from.c_str(), text.from.size, pos, endpos, QIO_REGEXP_ANCHOR_UNANCHORED, matches, nmatches);
      } else {
       got = qio_regexp_match(_regexp, text.c_str(), text.size, pos, endpos, QIO_REGEXP_ANCHOR_UNANCHORED, matches, nmatches);
      }
      // Now return where we matched.
      ret = new reMatch(got, matches[0].offset, matches[0].len);
      _ddata_free(matches);
    }
    return ret;
  }

  /** Check for a match to this regular expression at the start of the
      passed text. As with regexp.search, the capture group arguments
      here can be strings or any type that can reasonably cast from string. If
      a capture group was not matched, the corresponding argument will get the
      default value for its type.

      You can do
       if myregexp.match("some string") {
         ... do something if matched;
       }
   */
  proc match(text: ?t, ref captures ...?k):reMatch
    where t == string || t == stringPart
  {
    var ret:reMatch;
    on this.home {
      var pos:int;
      var endpos:int;
      if t == stringPart {
        pos = text.offset;
        endpos = pos + text.length;
      } else {
        pos = 0;
        endpos = text.size;
      }
      var matches:_ddata(qio_regexp_string_piece_t);
      var nmatches = 1 + captures.size;
      matches = _ddata_allocate(qio_regexp_string_piece_t, nmatches);
      var got:bool;
      if t == stringPart {
        got = qio_regexp_match(_regexp, text.from.c_str(), text.from.size, pos, endpos, QIO_REGEXP_ANCHOR_START, matches, nmatches);
      } else {
        got = qio_regexp_match(_regexp, text.c_str(), text.size, pos, endpos, QIO_REGEXP_ANCHOR_START, matches, nmatches);
      }
      // Now try to coerce the read strings into the captures.
      _handle_captures(text, matches, nmatches, captures);
      // Now return where we matched.
      ret = new reMatch(got, matches[0].offset, matches[0].len);
      _ddata_free(matches);
    }
    return ret;
  }
  proc match(text: ?t):reMatch
    where t == string || t == stringPart
  {
    var ret:reMatch;
    on this.home {
      var pos:int;
      var endpos:int;
      if t == stringPart {
        pos = text.offset;
        endpos = pos + text.length;
      } else {
        pos = 0;
        endpos = text.size;
      }
      var matches:_ddata(qio_regexp_string_piece_t);
      var nmatches = 1;
      matches = _ddata_allocate(qio_regexp_string_piece_t, nmatches);
      var got:bool;
      if t == stringPart {
       got = qio_regexp_match(_regexp, text.from.c_str(), text.from.size, pos, endpos, QIO_REGEXP_ANCHOR_START, matches, nmatches);
      } else {
       got = qio_regexp_match(_regexp, text.c_str(), text.size, pos, endpos, QIO_REGEXP_ANCHOR_START, matches, nmatches);
      }
      // Now return where we matched.
      ret = new reMatch(got, matches[0].offset, matches[0].len);
      _ddata_free(matches);
    }
    return ret;
  }

  /** Split the text by occurrences of this regular expression.
     If capturing parentheses are used in pattern, then the text of all
     groups in the pattern are also returned as part of the resulting array.
     If maxsplit is nonzero, at most maxsplit splits occur, and the
     remaining text is returned as the last element.
   */
  iter split(text: ?t, maxsplit: int = 0)
    where t == string || t == stringPart
  {
    var matches:_ddata(qio_regexp_string_piece_t);
    var ncaptures = qio_regexp_get_ncaptures(_regexp);
    var nmatches = 1 + ncaptures;
    var pos:int;
    var endpos:int;

    on this.home {
      matches = _ddata_allocate(qio_regexp_string_piece_t, nmatches);
    }

    if t == stringPart {
      pos = text.offset;
      endpos = pos + text.length;
    } else {
      pos = 0;
      endpos = text.size;
    }
    var splits = 0;
    var maxsplits = maxsplit;
    if maxsplit == 0 then maxsplits = max(int);

    while true {
      var splitstart = 0;
      var splitend = 0;
      var got:bool;
      on this.home {
        got = qio_regexp_match(_regexp, text.c_str(), text.size, pos, endpos, QIO_REGEXP_ANCHOR_UNANCHORED, matches, nmatches);
      }

      splits += 1;
      if got && splits <= maxsplits {
        splitstart = matches[0].offset;
        splitend = matches[0].offset + matches[0].len;
      } else {
        splitstart = endpos;
        splitend = endpos;
      }

      if pos < endpos {
        // Yield splitted value
        yield text.substring(pos+1..splitstart);
      } else {
        yield "";
      }

      if got {
        // Yield capture groups
        for i in 1..ncaptures {
          yield text.substring(new reMatch(
                !qio_regexp_string_piece_isnull(matches[i]),
                matches[i].offset,
                matches[i].len));
        }
      }

      // Advance to splitend.
      pos = splitend;

      if splits > maxsplits || !got then break;
    }
    on this.home {
      _ddata_free(matches);
    }
  }
  /* Enumerates matches in the string as well as capture groups.
     Returns tuples of reMatch objects, the 1st is always
      the match for the whole pattern and the rest are the capture groups.
   */
  iter matches(text: ?t, param captures=0, maxmatches: int = max(int))
    where t == string || t == stringPart
  {
    var matches:_ddata(qio_regexp_string_piece_t);
    var nmatches = 1 + captures;
    var pos:int;
    var endpos:int;
    on this.home {
      matches = _ddata_allocate(qio_regexp_string_piece_t, nmatches);
    }
    if t == stringPart {
      pos = text.offset;
      endpos = pos + text.length;
    } else {
      pos = 0;
      endpos = text.size;
    }
    var nfound = 0;
    var cur = pos;
    while nfound < maxmatches && cur < endpos {
      var got:bool;
      on this.home {
        got = qio_regexp_match(_regexp, text.c_str(), text.size, cur, endpos, QIO_REGEXP_ANCHOR_UNANCHORED, matches, nmatches);
      }
      if !got then break;
      param nret = captures+1;
      var ret:nret*reMatch;
      for i in 0..captures {
        ret[i+1] = new reMatch(got, matches[i].offset, matches[i].len);
      }
      yield ret;
      cur = matches[0].offset + matches[0].len;
    }
    on this.home {
      _ddata_free(matches);
    }
  }
  proc subn(repl:string, text: ?t, global = true ):(string, int)
    where t == string || t == stringPart
  {
    var pos:int;
    var endpos:int;
    if t == stringPart {
      pos = text.offset;
      endpos = pos + text.length;
    } else {
      pos = 0;
      endpos = text.size;
    }
    var replaced:c_string; // FIX ME: leak c_string
    var nreplaced:int;
    var replaced_len:int(64);
    if t == stringPart {
      nreplaced = qio_regexp_replace(_regexp, repl.c_str(), repl.size, text.from.c_str(), text.from.size, pos, endpos, global, replaced, replaced_len);
    } else {
      nreplaced = qio_regexp_replace(_regexp, repl.c_str(), repl.size, text.c_str(), text.size, pos, endpos, global, replaced, replaced_len);
    }
    return (toString(replaced), nreplaced);
  }
  proc sub(repl:string, text: ?t, global = true )
    where t == string || t == stringPart
  {
    var (str, count) = subn(repl, text, global);
    return str;
  }
  proc writeThis(f: Writer) {
    var pattern:c_string;
    on this.home {
      qio_regexp_get_pattern(this._regexp, pattern);
    }
    // Note -- this is wrong because we didn't quote
    // and there's no way to get the flags
    f.write("new regexp(\"", pattern, "\")");
  }
  proc readThis(f: Reader) {
    var pattern:c_string;
    // Note -- this is wrong because we didn't quote
    // and there's no way to get the flags
    var litOne = new ioLiteral("new regexp(\"");
    var litTwo = new ioLiteral("\")");
    if(f.read(litOne, pattern, litTwo)) {
      on this.home {
        var opts:qio_regexp_options_t;
        qio_regexp_init_default_options(opts);
        qio_regexp_create_compile(pattern, pattern.size, opts, this._regexp);
      }
    }
  }
}

proc =(ref ret:regexp, x:regexp)
{
  // retain -- release
  if x.home == ret.home {
    on x.home {
      qio_regexp_retain(x._regexp);
      qio_regexp_release(ret._regexp);
    }
    ret._regexp = x._regexp;
  } else {
    var pattern:c_string;
    var options:qio_regexp_options_t;

    on ret.home {
      qio_regexp_release(ret._regexp);
    }
    on x.home {
      qio_regexp_get_pattern(x._regexp, pattern);
      qio_regexp_get_options(x._regexp, options);
    }

    qio_regexp_create_compile(pattern, pattern.size, options, ret._regexp);
  }
}

// TODO -- shouldn't have to write this this way!
pragma "init copy fn"
proc chpl__initCopy(x: regexp) {
  on x.home {
    qio_regexp_retain(x._regexp);
  }
  return x;
}

// Cast regexp to string.
inline proc _cast(type t, x: regexp) where t == string {
  var pattern:c_string;
  on x.home {
    qio_regexp_get_pattern(x._regexp, pattern);
  }
  // FIX ME: leak c_string
  return toString(pattern);
}
// Cast string to regexp
inline proc _cast(type t, x: string) where t == regexp {
  return compile(x);
}

// string methods for searching
proc string.search(needle: string, ignorecase=false):reMatch
{
  // Create a regexp matching the literal for needle
  var re = compile(needle, literal=true, nocapture=true, ignorecase=ignorecase);
  return re.search(this);
}
proc string.search(needle: regexp):reMatch
{
  return needle.search(this);
}
proc string.search(needle: regexp, ref captures ...?k):reMatch
{
  return needle.search(this, (...captures));
}
proc string.match(pattern: regexp):reMatch
{
  return pattern.match(this);
}
proc string.match(pattern: regexp, ref captures ...?k):reMatch
{
  return pattern.match(this, (...captures));
}

iter string.split(pattern: regexp, maxsplit: int = 0)
{
  for v in pattern.split(this, maxsplit) {
    yield v;
  }
}

iter string.matches(pattern:regexp, param captures=0, maxmatches:int=max(int))
{
  for v in pattern.matches(this, captures, maxmatches) {
    yield v;
  }
}


This version of Random.chpl does not have its first line bold

I'm not sure what the difference is.

/*
 * Copyright 2004-2015 Cray Inc.
 * Other additional copyright holders may be indicated within.
 *
 * The entirety of this work is licensed under the Apache License,
 * Version 2.0 (the "License"); you may not use this file except
 * in compliance with the License.
 *
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

/*
   Support for pseudorandom number generation

   This module defines an abstraction for a stream of pseudorandom
   numbers, :chpl:class:RandomStream:.  The current implementation is
   based on the one used in the NAS Parallel Benchmarks (NPB,
   available at: http://www.nas.nasa.gov/publications/npb.html).  The
   longer-term intention is to add knobs permitting users to select
   other pseudorandom number generation algorithms, such as the
   Mersenne twister.

   Paraphrasing the comments from the NPB reference implementation:

     This generator returns uniform pseudorandom real values in the
     range (0, 1) by using the linear congruential generator

       x_{k+1} = a x_k  (mod 2**46)

     where 0 < x_k < 2**46 and 0 < a < 2**46.  This scheme generates
     2**44 numbers before repeating.  The seed value must be an odd
     64-bit integer in the range (1, 2^46).  The generated values are
     normalized to be between 0 and 1, i.e., 2**(-46) * x_k.

     This generator should produce the same results on any computer
     with at least 48 mantissa bits for real(64) data.

   Here is a list of currently open issues (TODOs) for this module:

   1. We plan to support general serial and parallel iterators on the
   RandomStream class; however, providing the full suite of iterators
   is not possible with our current parallel iterator framework.
   Specifically, if :chpl:class:RandomStream: is a follower in a
   zippered iteration context, there is no way for it to update the
   total number of random numbers generated in a safe/sane/coordinated
   way.  We are exploring a revised leader-follower iterator framework
   that would support this idiom (and other cursor-based ones).

   2. This module is currently restricted to generating real(64),
   imag(64), and complex(128) complex values.  We would like to extend
   this support to include other primitive types as well.

   3. If no seed is provided by the user, one is chosen based on the
   current time in microseconds, allowing for some degree of
   pseudorandomness in seed selection.  The intent of the
   SeedGenerator record is to provide a menu of other options for
   initializing the random stream seed, but only one option is
   implemented at present.

   3. Can the multiplier 'arand' be moved into the RandomStream class
   so that it can be changed by a user of this class?

*/
module Random {

  /* Note to developers on "private" symbols:

     It is the intent that once Chapel supports a notion of 'private'
     symbols, everything prefixed with RandomPrivate will be made
     private to this module and everything prefixed with
     'RandomStreamPrivate_' will be made private to the RandomStream
     class.
  */

/* TODO: Document fillRandom here */
proc fillRandom(x:[], seed: int(64)) {
  if x.eltType != complex && x.eltType != real && x.eltType != imag then
    compilerError("Random.fillRandom is only defined for real(64), imag(64), and complex(128) arrays");
  var randNums = new RandomStream(seed, parSafe=false);
  randNums.fillRandom(x);
  delete randNums;
}

proc fillRandom(x:[]) {
  if x.eltType != complex && x.eltType != real && x.eltType != imag then
    compilerError("Random.fillRandom is only defined for real(64), imag(64), and complex(128) arrays");
  var randNums = new RandomStream(parSafe=false);
  randNums.fillRandom(x);
  delete randNums;
}

record SeedGenerators {
  proc currentTime {
    use Time;
    const seed: int(64) = getCurrentTime(unit=TimeUnits.microseconds):int(64);
    return (if seed % 2 == 0 then seed + 1 else seed) % (1:int(64) << 46);
  }
};

const SeedGenerator: SeedGenerators;

class RandomStream {
  param parSafe: bool = true;
  const seed: int(64);

  proc RandomStream(seed: int(64) = SeedGenerator.currentTime,
                   param parSafe: bool = true) {
    if seed % 2 == 0 || seed < 1 || seed > 1:int(64)<<46 then
      halt("RandomStream seed must be an odd integer between 0 and 2**46");
    RandomStreamPrivate_init(seed);
  }

  proc getNext(param parSafe = this.parSafe) {
    if parSafe then
      RandomStreamPrivate_lock$ = true;
    RandomStreamPrivate_count += 1;
    const result = RandomPrivate_randlc(RandomStreamPrivate_cursor);
    if parSafe then
      RandomStreamPrivate_lock$;
    return result;
  }

  proc skipToNth(n: integral, param parSafe = this.parSafe) {
    if n <= 0 then
      halt("RandomStream.skipToNth(n) called with non-positive 'n' value", n);
    if parSafe then
      RandomStreamPrivate_lock$ = true;
    RandomStreamPrivate_count = n;
    RandomStreamPrivate_cursor = RandomPrivate_randlc_skipto(seed, n);
    if parSafe then
      RandomStreamPrivate_lock$;
  }

  proc getNth(n: integral, param parSafe = this.parSafe) {
    if (n <= 0) then
      halt("RandomStream.getNth(n) called with non-positive 'n' value", n);
    if parSafe then
      RandomStreamPrivate_lock$ = true;
    skipToNth(n, parSafe=false);
    const result = getNext(parSafe=false);
    if parSafe then
      RandomStreamPrivate_lock$;
    return result;
  }

  proc fillRandom(X: [], param parSafe = this.parSafe) {
    if X.eltType != complex && X.eltType != real && X.eltType != imag then
      compilerError("RandomStream.fillRandom is only defined for real(64), imag(64), and complex(128) arrays");
    forall (x, r) in zip(X, iterate(X.domain, X.eltType, parSafe)) do
      x = r;
  }

  proc iterate(D: domain, type resultType=real, param parSafe = this.parSafe) {
    if resultType != complex && resultType != real && resultType != imag then
      compilerError("RandomStream.iterate is only defined for real(64), imag(64), and complex(128) result types");
    param cplxMultiplier = if resultType == complex then 2 else 1;
    if parSafe then
      RandomStreamPrivate_lock$ = true;
    const start = RandomStreamPrivate_count;
    // NOTE: Not bothering to check to see if D.numIndices can fit into int(64)
    RandomStreamPrivate_count += cplxMultiplier * D.numIndices:int(64);
    skipToNth(RandomStreamPrivate_count, parSafe=false);
    if parSafe then
      RandomStreamPrivate_lock$;
    return RandomPrivate_iterate(resultType, D, seed, start);
  }

  proc writeThis(f: Writer) {
    f <~> "RandomStream(parSafe=";
    f <~> parSafe;
    f <~> ", seed = ";
    f <~> seed;
    f <~> ")";
  }

  ///////////////////////////////////////////////////////////// CLASS PRIVATE //
  //
  // It is the intent that once Chapel supports the notion of
  // 'private', everything in this class declared below this line will
  // be made private to this class.
  //

  var RandomStreamPrivate_lock$: sync bool;
  var RandomStreamPrivate_cursor: real;
  var RandomStreamPrivate_count: int(64);

  proc RandomStreamPrivate_init(seed: int(64)) {
    this.seed = seed;
    RandomStreamPrivate_cursor = seed;
    RandomStreamPrivate_count = 1;
  }
}

////////////////////////////////////////////////////////////// MODULE PRIVATE //
//
// It is the intent that once Chapel supports the notion of 'private',
// everything declared below this line will be made private to this
// module.
//

//
// NPB-defined constants for linear congruential generator
//
const RandomPrivate_r23   = 0.5**23,
      RandomPrivate_t23   = 2.0**23,
      RandomPrivate_r46   = 0.5**46,
      RandomPrivate_t46   = 2.0**46,
      RandomPrivate_arand = 1220703125.0; // TODO: Is arand something that a
                                          // user might want to set on a
                                          // case-by-case basis?

//
// NPB-defined randlc routine
//
proc RandomPrivate_randlc(inout x: real, a: real = RandomPrivate_arand) {
  var t1 = RandomPrivate_r23 * a;
  const a1 = floor(t1),
    a2 = a - RandomPrivate_t23 * a1;
  t1 = RandomPrivate_r23 * x;
  const x1 = floor(t1),
    x2 = x - RandomPrivate_t23 * x1;
  t1 = a1 * x2 + a2 * x1;
  const t2 = floor(RandomPrivate_r23 * t1),
    z  = t1 - RandomPrivate_t23 * t2,
    t3 = RandomPrivate_t23 * z + a2 * x2,
    t4 = floor(RandomPrivate_r46 * t3),
    x3 = t3 - RandomPrivate_t46 * t4;
  x = x3;
  return RandomPrivate_r46 * x3;
}

// Wrapper that takes a result type (two calls for complex types)
proc RandomPrivate_randlc(type resultType, inout x: real) {
  if resultType == complex then
    return (RandomPrivate_randlc(x), RandomPrivate_randlc(x)):complex;
  else
    return RandomPrivate_randlc(x):resultType;
}

//
// Return a value for the cursor so that the next call to randlc will
// return the same value as the nth call to randlc
//
proc RandomPrivate_randlc_skipto(seed: int(64), in n: integral): real {
  var cursor = seed:real;
  n -= 1;
  var t = RandomPrivate_arand;
  RandomPrivate_arand;
  while (n != 0) {
    const i = n / 2;
    if (2 * i != n) then
      RandomPrivate_randlc(cursor, t);
    if i == 0 then
      break;
    else
      n = i;
    RandomPrivate_randlc(t, t);
    n = i;
  }
  return cursor;
}

//
// iterate over outer ranges in tuple of ranges
//
iter RandomPrivate_outer(ranges, param dim: int = 1) {
  if dim + 1 == ranges.size {
    for i in ranges(dim) do
      yield (i,);
  } else if dim + 1 < ranges.size {
    for i in ranges(dim) do
      for j in RandomPrivate_outer(ranges, dim+1) do
        yield (i, (...j));
  } else {
    yield 0; // 1D case is a noop
  }
}

//
// RandomStream iterator implementation
//
iter RandomPrivate_iterate(type resultType, D: domain, seed: int(64),
                          start: int(64)) {
  var cursor = RandomPrivate_randlc_skipto(seed, start);
  for i in D do
    yield RandomPrivate_randlc(resultType, cursor);
}

iter RandomPrivate_iterate(type resultType, D: domain, seed: int(64),
                          start: int(64), param tag: iterKind)
      where tag == iterKind.leader {
  for block in D._value.these(tag=iterKind.leader) do
    yield block;
}

iter RandomPrivate_iterate(type resultType, D: domain, seed: int(64),
                          start: int(64), param tag: iterKind, followThis)
      where tag == iterKind.follower {
  param multiplier = if resultType == complex then 2 else 1;
  const ZD = computeZeroBasedDomain(D);
  const innerRange = followThis(ZD.rank);
  var cursor: real;
  for outer in RandomPrivate_outer(followThis) {
    var myStart = start;
    // NOTE: Not bothering to check to see if this can fit into int(64)
    if ZD.rank > 1 then
      myStart += multiplier * ZD.indexOrder(((...outer), innerRange.low)):int(64);
    else
      myStart += multiplier * ZD.indexOrder(innerRange.low):int(64);
    if !innerRange.stridable {
      cursor = RandomPrivate_randlc_skipto(seed, myStart);
      for i in innerRange do
        yield RandomPrivate_randlc(resultType, cursor);
    } else {
      // NOTE: Not bothering to check to see if this can fit into int(64)
      myStart -= innerRange.low:int(64);
      for i in innerRange {
        cursor = RandomPrivate_randlc_skipto(seed, myStart + i:int(64) * multiplier);
        yield RandomPrivate_randlc(resultType, cursor);
      }
    }
  }
}

} // close module Random

/*
 * Copyright 2004-2015 Cray Inc.
 * Other additional copyright holders may be indicated within.
 *
 * The entirety of this work is licensed under the Apache License,
 * Version 2.0 (the "License"); you may not use this file except
 * in compliance with the License.
 *
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

/*
   This module defines an abstraction for a stream of pseudorandom
   numbers, :chpl:class:RandomStream:.  The current implementation is
   based on the one used in the NAS Parallel Benchmarks (NPB,
   available at: http://www.nas.nasa.gov/publications/npb.html).  The
   longer-term intention is to add knobs permitting users to select
   other pseudorandom number generation algorithms, such as the
   Mersenne twister.

   Paraphrasing the comments from the NPB reference implementation:

     This generator returns uniform pseudorandom real values in the
     range (0, 1) by using the linear congruential generator

       x_{k+1} = a x_k  (mod 2**46)

     where 0 < x_k < 2**46 and 0 < a < 2**46.  This scheme generates
     2**44 numbers before repeating.  The seed value must be an odd
     64-bit integer in the range (1, 2^46).  The generated values are
     normalized to be between 0 and 1, i.e., 2**(-46) * x_k.

     This generator should produce the same results on any computer
     with at least 48 mantissa bits for real(64) data.

   Here is a list of currently open issues (TODOs) for this module:

   1. We plan to support general serial and parallel iterators on the
   RandomStream class; however, providing the full suite of iterators
   is not possible with our current parallel iterator framework.
   Specifically, if :chpl:class:RandomStream: is a follower in a
   zippered iteration context, there is no way for it to update the
   total number of random numbers generated in a safe/sane/coordinated
   way.  We are exploring a revised leader-follower iterator framework
   that would support this idiom (and other cursor-based ones).

   2. This module is currently restricted to generating real(64),
   imag(64), and complex(128) complex values.  We would like to extend
   this support to include other primitive types as well.

   3. If no seed is provided by the user, one is chosen based on the
   current time in microseconds, allowing for some degree of
   pseudorandomness in seed selection.  The intent of the
   SeedGenerator record is to provide a menu of other options for
   initializing the random stream seed, but only one option is
   implemented at present.

   3. Can the multiplier 'arand' be moved into the RandomStream class
   so that it can be changed by a user of this class?

*/
module Random {

  /* Note to developers on "private" symbols:

     It is the intent that once Chapel supports a notion of 'private'
     symbols, everything prefixed with RandomPrivate will be made
     private to this module and everything prefixed with
     'RandomStreamPrivate_' will be made private to the RandomStream
     class.
  */

/* TODO: Document fillRandom here */
proc fillRandom(x:[], seed: int(64)) {
  if x.eltType != complex && x.eltType != real && x.eltType != imag then
    compilerError("Random.fillRandom is only defined for real(64), imag(64), and complex(128) arrays");
  var randNums = new RandomStream(seed, parSafe=false);
  randNums.fillRandom(x);
  delete randNums;
}

proc fillRandom(x:[]) {
  if x.eltType != complex && x.eltType != real && x.eltType != imag then
    compilerError("Random.fillRandom is only defined for real(64), imag(64), and complex(128) arrays");
  var randNums = new RandomStream(parSafe=false);
  randNums.fillRandom(x);
  delete randNums;
}

record SeedGenerators {
  proc currentTime {
    use Time;
    const seed: int(64) = getCurrentTime(unit=TimeUnits.microseconds):int(64);
    return (if seed % 2 == 0 then seed + 1 else seed) % (1:int(64) << 46);
  }
};

const SeedGenerator: SeedGenerators;

class RandomStream {
  param parSafe: bool = true;
  const seed: int(64);

  proc RandomStream(seed: int(64) = SeedGenerator.currentTime,
                   param parSafe: bool = true) {
    if seed % 2 == 0 || seed < 1 || seed > 1:int(64)<<46 then
      halt("RandomStream seed must be an odd integer between 0 and 2**46");
    RandomStreamPrivate_init(seed);
  }

  proc getNext(param parSafe = this.parSafe) {
    if parSafe then
      RandomStreamPrivate_lock$ = true;
    RandomStreamPrivate_count += 1;
    const result = RandomPrivate_randlc(RandomStreamPrivate_cursor);
    if parSafe then
      RandomStreamPrivate_lock$;
    return result;
  }

  proc skipToNth(n: integral, param parSafe = this.parSafe) {
    if n <= 0 then
      halt("RandomStream.skipToNth(n) called with non-positive 'n' value", n);
    if parSafe then
      RandomStreamPrivate_lock$ = true;
    RandomStreamPrivate_count = n;
    RandomStreamPrivate_cursor = RandomPrivate_randlc_skipto(seed, n);
    if parSafe then
      RandomStreamPrivate_lock$;
  }

  proc getNth(n: integral, param parSafe = this.parSafe) {
    if (n <= 0) then
      halt("RandomStream.getNth(n) called with non-positive 'n' value", n);
    if parSafe then
      RandomStreamPrivate_lock$ = true;
    skipToNth(n, parSafe=false);
    const result = getNext(parSafe=false);
    if parSafe then
      RandomStreamPrivate_lock$;
    return result;
  }

  proc fillRandom(X: [], param parSafe = this.parSafe) {
    if X.eltType != complex && X.eltType != real && X.eltType != imag then
      compilerError("RandomStream.fillRandom is only defined for real(64), imag(64), and complex(128) arrays");
    forall (x, r) in zip(X, iterate(X.domain, X.eltType, parSafe)) do
      x = r;
  }

  proc iterate(D: domain, type resultType=real, param parSafe = this.parSafe) {
    if resultType != complex && resultType != real && resultType != imag then
      compilerError("RandomStream.iterate is only defined for real(64), imag(64), and complex(128) result types");
    param cplxMultiplier = if resultType == complex then 2 else 1;
    if parSafe then
      RandomStreamPrivate_lock$ = true;
    const start = RandomStreamPrivate_count;
    // NOTE: Not bothering to check to see if D.numIndices can fit into int(64)
    RandomStreamPrivate_count += cplxMultiplier * D.numIndices:int(64);
    skipToNth(RandomStreamPrivate_count, parSafe=false);
    if parSafe then
      RandomStreamPrivate_lock$;
    return RandomPrivate_iterate(resultType, D, seed, start);
  }

  proc writeThis(f: Writer) {
    f <~> "RandomStream(parSafe=";
    f <~> parSafe;
    f <~> ", seed = ";
    f <~> seed;
    f <~> ")";
  }

  ///////////////////////////////////////////////////////////// CLASS PRIVATE //
  //
  // It is the intent that once Chapel supports the notion of
  // 'private', everything in this class declared below this line will
  // be made private to this class.
  //

  var RandomStreamPrivate_lock$: sync bool;
  var RandomStreamPrivate_cursor: real;
  var RandomStreamPrivate_count: int(64);

  proc RandomStreamPrivate_init(seed: int(64)) {
    this.seed = seed;
    RandomStreamPrivate_cursor = seed;
    RandomStreamPrivate_count = 1;
  }
}

////////////////////////////////////////////////////////////// MODULE PRIVATE //
//
// It is the intent that once Chapel supports the notion of 'private',
// everything declared below this line will be made private to this
// module.
//

//
// NPB-defined constants for linear congruential generator
//
const RandomPrivate_r23   = 0.5**23,
      RandomPrivate_t23   = 2.0**23,
      RandomPrivate_r46   = 0.5**46,
      RandomPrivate_t46   = 2.0**46,
      RandomPrivate_arand = 1220703125.0; // TODO: Is arand something that a
                                          // user might want to set on a
                                          // case-by-case basis?

//
// NPB-defined randlc routine
//
proc RandomPrivate_randlc(inout x: real, a: real = RandomPrivate_arand) {
  var t1 = RandomPrivate_r23 * a;
  const a1 = floor(t1),
    a2 = a - RandomPrivate_t23 * a1;
  t1 = RandomPrivate_r23 * x;
  const x1 = floor(t1),
    x2 = x - RandomPrivate_t23 * x1;
  t1 = a1 * x2 + a2 * x1;
  const t2 = floor(RandomPrivate_r23 * t1),
    z  = t1 - RandomPrivate_t23 * t2,
    t3 = RandomPrivate_t23 * z + a2 * x2,
    t4 = floor(RandomPrivate_r46 * t3),
    x3 = t3 - RandomPrivate_t46 * t4;
  x = x3;
  return RandomPrivate_r46 * x3;
}

// Wrapper that takes a result type (two calls for complex types)
proc RandomPrivate_randlc(type resultType, inout x: real) {
  if resultType == complex then
    return (RandomPrivate_randlc(x), RandomPrivate_randlc(x)):complex;
  else
    return RandomPrivate_randlc(x):resultType;
}

//
// Return a value for the cursor so that the next call to randlc will
// return the same value as the nth call to randlc
//
proc RandomPrivate_randlc_skipto(seed: int(64), in n: integral): real {
  var cursor = seed:real;
  n -= 1;
  var t = RandomPrivate_arand;
  RandomPrivate_arand;
  while (n != 0) {
    const i = n / 2;
    if (2 * i != n) then
      RandomPrivate_randlc(cursor, t);
    if i == 0 then
      break;
    else
      n = i;
    RandomPrivate_randlc(t, t);
    n = i;
  }
  return cursor;
}

//
// iterate over outer ranges in tuple of ranges
//
iter RandomPrivate_outer(ranges, param dim: int = 1) {
  if dim + 1 == ranges.size {
    for i in ranges(dim) do
      yield (i,);
  } else if dim + 1 < ranges.size {
    for i in ranges(dim) do
      for j in RandomPrivate_outer(ranges, dim+1) do
        yield (i, (...j));
  } else {
    yield 0; // 1D case is a noop
  }
}

//
// RandomStream iterator implementation
//
iter RandomPrivate_iterate(type resultType, D: domain, seed: int(64),
                          start: int(64)) {
  var cursor = RandomPrivate_randlc_skipto(seed, start);
  for i in D do
    yield RandomPrivate_randlc(resultType, cursor);
}

iter RandomPrivate_iterate(type resultType, D: domain, seed: int(64),
                          start: int(64), param tag: iterKind)
      where tag == iterKind.leader {
  for block in D._value.these(tag=iterKind.leader) do
    yield block;
}

iter RandomPrivate_iterate(type resultType, D: domain, seed: int(64),
                          start: int(64), param tag: iterKind, followThis)
      where tag == iterKind.follower {
  param multiplier = if resultType == complex then 2 else 1;
  const ZD = computeZeroBasedDomain(D);
  const innerRange = followThis(ZD.rank);
  var cursor: real;
  for outer in RandomPrivate_outer(followThis) {
    var myStart = start;
    // NOTE: Not bothering to check to see if this can fit into int(64)
    if ZD.rank > 1 then
      myStart += multiplier * ZD.indexOrder(((...outer), innerRange.low)):int(64);
    else
      myStart += multiplier * ZD.indexOrder(innerRange.low):int(64);
    if !innerRange.stridable {
      cursor = RandomPrivate_randlc_skipto(seed, myStart);
      for i in innerRange do
        yield RandomPrivate_randlc(resultType, cursor);
    } else {
      // NOTE: Not bothering to check to see if this can fit into int(64)
      myStart -= innerRange.low:int(64);
      for i in innerRange {
        cursor = RandomPrivate_randlc_skipto(seed, myStart + i:int(64) * multiplier);
        yield RandomPrivate_randlc(resultType, cursor);
      }
    }
  }
}

} // close module Random