
RandomData
<dd>
<source>
RandomData randomData = new RandomDataImpl(new RngPackGenerator());
</source>
</dd>
<dt>Create an adaptor instance based on the Mersenne Twister generator
that can be used in place of a <code>Random
<dd>
<source>
RandomGenerator generator = new RngPackGenerator();
Random random = RandomAdaptor.createAdaptor(generator);
// random can now be used in place of a Random instance, data generation
// calls will be delegated to the wrapped Mersenne Twister
</source>
</dd>
</dl>
</p>
</subsection>
</section>
</body>
</document>

Java example source code file (random.xml)
The random.xml Java example source code<?xml version="1.0"?> <! Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. > <?xmlstylesheet type="text/xsl" href="./xdoc.xsl"?> <document url="random.html"> <properties> <title>The Commons Math User Guide  Data Generation </properties> <body> <section name="2 Data Generation"> <subsection name="2.1 Overview" href="overview"> <p> The Commons Math random package includes utilities for <ul> <li>generating random numbers <li>generating random vectors <li>generating random strings <li>generating cryptographically secure sequences of random numbers or strings</li> <li>generating random samples and permutations <li>analyzing distributions of values in an input file and generating values "like" the values in the file</li> <li>generating data for grouped frequency distributions or histograms</li> </ul> <p> The source of random data used by the data generation utilities is pluggable. By default, the JDKsupplied PseudoRandom Number Generator (PRNG) is used, but alternative generators can be "plugged in" using an adaptor framework, which provides a generic facility for replacing <code>java.util.Random with an alternative PRNG. Other very good PRNG suitable for MonteCarlo analysis (but <strong>not for cryptography) provided by the library are the Mersenne twister from Makoto Matsumoto and Takuji Nishimura and the more recent WELL generators (Well Equidistributed Longperiod Linear) from François Panneton, Pierre L'Ecuyer and Makoto Matsumoto. </p> <p> Sections 2.22.6 below show how to use the commons math API to generate different kinds of random data. The examples all use the default JDKsupplied PRNG. PRNG pluggability is covered in 2.7. The only modification required to the examples to use alternative PRNGs is to replace the argumentless constructor calls with invocations including a <code>RandomGenerator instance as a parameter. </p> </subsection> <subsection name="2.2 Random numbers" href="deviates"> <p> The <a href="../apidocs/org/apache/commons/math3/random/RandomDataGenerator.html"> RandomDataGenerator</a> class implements methods for generating random sequences of numbers. The API contracts of these methods use the following concepts: <dl> <dt>Random sequence of numbers from a probability distribution <dd>There is no such thing as a single "random number." What can be generated are <i>sequences of numbers that appear to be random. When using the builtin JDK function <code>Math.random(), sequences of values generated follow the <a href="http://www.itl.nist.gov/div898/handbook/eda/section3/eda3662.htm"> Uniform Distribution</a>, which means that the values are evenly spread over the interval between 0 and 1, with no subinterval having a greater probability of containing generated values than any other interval of the same length. The mathematical concept of a <a href="http://www.itl.nist.gov/div898/handbook/eda/section3/eda36.htm"> probability distribution</a> basically amounts to asserting that different ranges in the set of possible values of a random variable have different probabilities of containing the value. Commons Math supports generating random sequences from each of the distributions in the <a href="../apidocs/org/apache/commons/math3/distribution/packagesummary.html"> distributions</a> package. The javadoc for the <code>nextXxx methods in <a href="../apidocs/org/apache/commons/math3/random/RandomDataGenerator.html"> RandomDataGenerator</a> describes the algorithms used to generate random deviates. </dd> <dt>Cryptographically secure random sequences <dd>It is possible for a sequence of numbers to appear random, but nonetheless to be predictable based on the algorithm used to generate the sequence. If in addition to randomness, strong unpredictability is required, it is best to use a <a href="http://www.wikipedia.org/wiki/Cryptographically_secure_pseudorandom_number_generator"> secure random number generator</a> to generate values (or strings). The nextSecureXxx methods implemented by <code>RandomDataGenerator use the JDK <code>SecureRandom PRNG to generate cryptographically secure sequences. The <code>setSecureAlgorithm method allows you to change the underlying PRNG. These methods are <strong>much slower than the corresponding "nonsecure" versions, so they should only be used when cryptographic security is required. The <code>ISAACRandom class implements a fast cryptographically secure pseudorandom number generator.</dd> <dt>Seeding pseudorandom number generators <dd>By default, the implementation provided in
nextSecureHexString
can be used to generate random strings of hexadecimal characters. Both
of these methods produce sequences of strings with good dispersion
properties. The difference between the two methods is that the second is
cryptographically secure. Specifically, the implementation of
<code>nextHexString(n) in RandomDataGenerator uses the
following simple algorithm to generate a string of <code>n hex digits:
<ol>
<li>n/2+1 binary bytes are generated using the underlying RandomGenerator
<li>Each binary byte is translated into 2 hex digits
The <code>RandomDataGenerator implementation of the "secure" version,
<code>nextSecureHexString generates hex characters in 40byte
"chunks" using a 3step process:
<ol>
<li>20 random bytes are generated using the underlying
<code>SecureRandom.
<li>SHA1 hash is applied to yield a 20byte binary digest.
<li>Each byte of the binary digest is converted to 2 hex digits
Similarly to the secure random number generation methods,
<code>nextSecureHexString is much slower than
the nonsecure version. It should be used only for applications such as
generating unique session or transaction ids where predictability of
subsequent ids based on observation of previous values is a security
concern. If all that is needed is an even distribution of hex characters
in the generated strings, the nonsecure method should be used.
</p>
</subsection>
<subsection name="2.5 Random permutations, combinations, sampling"
href="combinatorics">
<p>
To select a random sample of objects in a collection, you can use the
<code>nextSample method implemented by RandomDataGenerator .
Specifically, if <code>c is a collection containing at least
<code>k objects, and randomData is a
<code>RandomDataGenerator instance randomData.nextSample(c, k)
will return an <code>object[] array of length k
consisting of elements randomly selected from the collection. If
<code>c contains duplicate references, there may be duplicate
references in the returned array; otherwise returned elements will be
unique  i.e., the sampling is without replacement among the object
references in the collection. </p>
<p>
If <code>randomData is a RandomDataGenerator instance, and
<code>n and k are integers with
<code> k <= n, then
<code>randomData.nextPermutation(n, k) returns an int[]
array of length <code>k whose whose entries are selected randomly,
without repetition, from the integers <code>0 through
<code>n1 (inclusive), i.e.,
<code>randomData.nextPermutation(n, k) returns a random
permutation of <code>n taken k at a time.
</p>
</subsection>
<subsection name="2.6 Generating data 'like' an input file" href="empirical">
<p>
Using the <code>ValueServer class, you can generate data based on
the values in an input file in one of two ways:
<dl>
<dt>Replay Mode
<dd> The following code will read data from url
(a <code>java.net.URL instance), cycling through the values in the
file in sequence, reopening and starting at the beginning again when all
values have been read.
<source>
ValueServer vs = new ValueServer();
vs.setValuesFileURL(url);
vs.setMode(ValueServer.REPLAY_MODE);
vs.resetReplayFile();
double value = vs.getNext();
// ...Generate and use more values...
vs.closeReplayFile();
</source>
The values in the file are not stored in memory, so it does not matter
how large the file is, but you do need to explicitly close the file
as above. The expected file format is \n delimited (i.e. one per line)
strings representing valid floating point numbers.
</dd>
<dt>Digest Mode
<dd>When used in Digest Mode, the ValueServer reads the entire input file
and estimates a probability density function based on data from the file.
The estimation method is essentially the
<a href="http://nedwww.ipac.caltech.edu/level5/March02/Silverman/Silver2_6.html">
Variable Kernel Method</a> with Gaussian smoothing. Once the density
has been estimated, <code>getNext() returns random values whose
probability distribution matches the empirical distribution  i.e., if
you generate a large number of such values, their distribution should
"look like" the distribution of the values in the input file. The values
are not stored in memory in this case either, so there is no limit to the
size of the input file. Here is an example:
<source>
ValueServer vs = new ValueServer();
vs.setValuesFileURL(url);
vs.setMode(ValueServer.DIGEST_MODE);
vs.computeDistribution(500); //Read file and estimate distribution using 500 bins
double value = vs.getNext();
// ...Generate and use more values...
</source>
See the javadoc for <code>ValueServer and
<code>EmpiricalDistribution for more details. Note that
<code>computeDistribution() opens and closes the input file
by itself.
</dd>
</dl>
</p>
</subsection>
<subsection name="2.7 PRNG Pluggability" href="pluggability">
<p>
To enable alternative PRNGs to be "plugged in" to the commonsmath data
generation utilities and to provide a generic means to replace
<code>java.util.Random in applications, a random generator
adaptor framework has been added to commonsmath. The
<a href="../apidocs/org/apache/commons/math3/random/RandomGenerator.html">
RandomGenerator</a> interface abstracts the public interface of
<code>java.util.Random and any implementation of this
interface can be used as the source of random data for the commonsmath
data generation classes. An abstract base class,
<a href="../apidocs/org/apache/commons/math3/random/AbstractRandomGenerator.html">
AbstractRandomGenerator</a> is provided to make implementation easier.
This class provides default implementations of "derived" data generation
methods based on the primitive, <code>nextDouble().
To support generic replacement of <code>java.util.Random, the
<a href="../apidocs/org/apache/commons/math3/random/RandomAdaptor.html">
RandomAdaptor</a> class is provided, which extends
<code>java.util.Random and wraps and delegates calls to
a <code>RandomGenerator instance.
</p>
<p>Commonsmath provides by itself several implementations of the JDKRandomGenerator
that extends the JDK provided generator</li>
<li>
AbstractRandomGenerator</a> as a helper for users generators
<li>
BitStreamGenerator</a> which is an abstract class for several generators and
which in turn is extended by:
<ul>
<li>MersenneTwister
<li>Well512a
<li>Well1024a
<li>Well19937a
<li>Well19937c
<li>Well44497a
<li>Well44497b
</ul>
</li>
</ul>
</p>
<p>
The JDK provided generator is a simple one that can be used only for very simple needs.
The Mersenne Twister is a fast generator with very good properties well suited for
MonteCarlo simulation. It is equidistributed for generating vectors up to dimension 623
and has a huge period: 2<sup>19937  1 (which is a Mersenne prime). This generator
is described in a paper by Makoto Matsumoto and Takuji Nishimura in 1998: <a
href="http://www.math.sci.hiroshimau.ac.jp/~mmat/MT/ARTICLES/mt.pdf">Mersenne Twister:
A 623Dimensionally Equidistributed Uniform PseudoRandom Number Generator</a>, ACM
Transactions on Modeling and Computer Simulation, Vol. 8, No. 1, January 1998, pp 330.
The WELL generators are a family of generators with period ranging from 2<sup>512  1
to 2<sup>44497  1 (this last one is also a Mersenne prime) with even better properties
than Mersenne Twister. These generators are described in a paper by François Panneton,
Pierre L'Ecuyer and Makoto Matsumoto <a
href="http://www.iro.umontreal.ca/~lecuyer/myftp/papers/wellrng.pdf">Improved LongPeriod
Generators Based on Linear Recurrences Modulo 2</a> ACM Transactions on Mathematical Software,
32, 1 (2006). The errata for the paper are in <a
href="http://www.iro.umontreal.ca/~lecuyer/myftp/papers/wellrngerrata.txt">wellrngerrata.txt</a>.
</p>
<p>
For simple sampling, any of these generators is sufficient. For MonteCarlo simulations the
JDK generator does not have any of the good mathematical properties of the other generators,
so it should be avoided. The Mersenne twister and WELL generators have equidistribution properties
proven according to their bits pool size which is directly linked to their period (all of them
have maximal period, i.e. a generator with size n pool has a period 2<sup>n1). They also
have equidistribution properties for 32 bits blocks up to s/32 dimension where s is their pool size.
So WELL19937c for exemple is equidistributed up to dimension 623 (19937/32). This means a MonteCarlo
simulation generating a vector of n variables at each iteration has some guarantees on the properties
of the vector as long as its dimension does not exceed the limit. However, since we use bits from two
successive 32 bits generated integers to create one double, this limit is smaller when the variables are
of type double. so for MonteCarlo simulation where less the 16 doubles are generated at each round,
WELL1024 may be sufficient. If a larger number of doubles are needed a generator with a larger pool
would be useful.
</p>
<p>
The WELL generators are more modern then MersenneTwister (the paper describing than has been published
in 2006 instead of 1998) and fix some of its (few) drawbacks. If initialization array contains many
zero bits, MersenneTwister may take a very long time (several hundreds of thousands of iterations to
reach a steady state with a balanced number of zero and one in its bits pool). So the WELL generators
are better to <i>escape zeroland as explained by the WELL generators creators. The Well19937a and
Well44497a generator are not maximally equidistributed (i.e. there are some dimensions or bits blocks
size for which they are not equidistributed). The Well512a, Well1024a, Well19937c and Well44497b are
maximally equidistributed for blocks size up to 32 bits (they should behave correctly also for double
based on more than 32 bits blocks, but equidistribution is not proven at these blocks sizes).
</p>
<p>
The MersenneTwister generator uses a 624 elements integer array, so it consumes less than 2.5 kilobytes.
The WELL generators use 6 integer arrays with a size equal to the pool size, so for example the
WELL44497b generator uses about 33 kilobytes. This may be important if a very large number of
generator instances were used at the same time.
</p>
<p>
All generators are quite fast. As an example, here are some comparisons, obtained on a 64 bits JVM on a
linux computer with a 2008 processor (AMD phenom Quad 9550 at 2.2 GHz). The generation rate for
MersenneTwister was between 25 and 27 millions doubles per second (remember we generate two 32 bits integers for
each double). Generation rates for other PRNG, relative to MersenneTwister:
</p>
<p>
<table border="1" align="center">
<tr BGCOLOR="#CCCCFF">  Example of performances  
Name  generation rate (relative to MersenneTwister)  
MersenneTwister  1  
JDKRandomGenerator  between 0.96 and 1.16  
Well512a  between 0.85 and 0.88  
Well1024a  between 0.63 and 0.73  
Well19937a  between 0.70 and 0.71  
Well19937c  between 0.57 and 0.71  
Well44497a  between 0.69 and 0.71  
Well44497b  between 0.65 and 0.71 
Copyright 19982019 Alvin Alexander, alvinalexander.com
All Rights Reserved.
A percentage of advertising revenue from
pages under the /java/jwarehouse
URI on this website is
paid back to open source projects.