| SPARK-5121 |
Stored as parquet doens't support the CTAS |
Major |
SQL |
XiaoJing wang |
|
| SPARK-4785 |
When called with arguments referring column fields, PMOD throws NPE |
Blocker |
SQL |
Cheng Lian |
|
| SPARK-4774 |
Make HiveFromSpark example more portable |
Major |
Examples, SQL |
Kostas Sakellis |
Kostas Sakellis |
| SPARK-4769 |
CTAS does not work when reading from temporary tables |
Critical |
SQL |
Michael Armbrust |
Cheng Hao |
| SPARK-4761 |
With JDBC server, set Kryo as default serializer and disable reference tracking |
Blocker |
SQL |
Patrick Wendell |
Cheng Lian |
| SPARK-4757 |
Yarn-client failed to start due to Wrong FS error in distCacheMgr.addResource |
Major |
YARN |
Jianshi Huang |
|
| SPARK-4753 |
Parquet2 does not prune based on OR filters on partition columns |
Blocker |
SQL |
Michael Armbrust |
Michael Armbrust |
| SPARK-4715 |
ShuffleMemoryManager.tryToAcquire may return a negative value |
Major |
Spark Core |
Shixiong Zhu |
Shixiong Zhu |
| SPARK-4701 |
Typo in sbt/sbt |
Trivial |
Build |
Masayoshi TSUZUKI |
Masayoshi TSUZUKI |
| SPARK-4686 |
Link to "allowed master URLs" is broken in configuration documentation |
Minor |
Documentation |
Kay Ousterhout |
Kay Ousterhout |
| SPARK-4676 |
JavaSchemaRDD.schema may throw NullType MatchError if sql has null |
Major |
SQL |
YanTang Zhai |
|
| SPARK-4672 |
Cut off the super long serialization chain in GraphX to avoid the StackOverflow error |
Critical |
GraphX, Spark Core |
Lijie Xu |
|
| SPARK-4670 |
bitwise NOT has a wrong toString output |
Major |
SQL |
Adrian Wang |
|
| SPARK-4668 |
Fix documentation typos |
Major |
Documentation |
Ryan Williams |
Ryan Williams |
| SPARK-4663 |
close() function is not surrounded by finally in ParquetTableOperations.scala |
Minor |
SQL |
baishuo |
|
| SPARK-4658 |
Code documentation issue in DDL of datasource |
Minor |
SQL |
Ravindra Pesala |
|
| SPARK-4650 |
Supporting multi column support in countDistinct function like count(distinct c1,c2..) in Spark SQL |
Major |
SQL |
Ravindra Pesala |
|
| SPARK-4645 |
Asynchronous execution in HiveThriftServer2 with Hive 0.13.1 doesn't play well with Simba ODBC driver |
Blocker |
SQL |
Cheng Lian |
Cheng Lian |
| SPARK-4627 |
Too many TripletFields |
Trivial |
GraphX |
Joseph E. Gonzalez |
|
| SPARK-4626 |
NoSuchElementException in CoarseGrainedSchedulerBackend |
Major |
Spark Core |
Victor Tso |
Victor Tso |
| SPARK-4619 |
Double "ms" in ShuffleBlockFetcherIterator log |
Minor |
. |
maji2014 |
maji2014 |
| SPARK-4615 |
Cannot disconnect from spark-shell |
Minor |
Spark Shell |
Thomas Omans |
|
| SPARK-4602 |
saveAsNewAPIHadoopFiles by default does not use SparkContext's hadoop configuration |
Major |
Streaming |
Tathagata Das |
Tathagata Das |
| SPARK-4601 |
Call site of jobs generated by streaming incorrect in Spark UI |
Major |
Streaming |
Tathagata Das |
Tathagata Das |
| SPARK-4593 |
sum(1/0) would produce a very large number |
Minor |
SQL |
Adrian Wang |
|
| SPARK-4592 |
"Worker registration failed: Duplicate worker ID" error during Master failover |
Blocker |
Deploy, Spark Core |
Josh Rosen |
Andrew Or |
| SPARK-4584 |
2x Performance regression for Spark-on-YARN |
Blocker |
YARN |
Nishkam Ravi |
Marcelo Vanzin |
| SPARK-4578 |
Row.asDict() should keep the type of values |
Blocker |
PySpark, SQL |
Davies Liu |
Davies Liu |
| SPARK-4552 |
query for empty parquet table in spark sql hive get IllegalArgumentException |
Blocker |
SQL |
Fei Wang |
Michael Armbrust |
| SPARK-4548 |
Python broadcast perf regression from Spark 1.1 |
Blocker |
PySpark |
Davies Liu |
Davies Liu |
| SPARK-4546 |
Improve HistoryServer first time user experience |
Critical |
Spark Core |
Andrew Or |
Andrew Or |
| SPARK-4536 |
Add sqrt and abs to Spark SQL DSL |
Major |
SQL |
Kousuke Saruta |
Kousuke Saruta |
| SPARK-4535 |
Fix the error in comments |
Minor |
Streaming |
Yadong Qi |
|
| SPARK-4532 |
make-distribution in Spark 1.2 does not correctly detect whether Hive is enabled |
Blocker |
Build |
Patrick Wendell |
Patrick Wendell |
| SPARK-4530 |
GradientDescent get a wrong gradient value according to the gradient formula, which is caused by the miniBatchSize parameter. |
Minor |
MLlib |
Guoqiang Li |
Guoqiang Li |
| SPARK-4525 |
MesosSchedulerBackend.resourceOffers cannot decline unused offers from acceptedOffers |
Blocker |
Mesos |
Jongyoul Lee |
Jongyoul Lee |
| SPARK-4519 |
Filestream does not use hadoop configuration set within sparkContext.hadoopConfiguration |
Major |
Streaming |
Tathagata Das |
Tathagata Das |
| SPARK-4518 |
Filestream sometimes processes files twice |
Blocker |
Streaming |
Tathagata Das |
Tathagata Das |
| SPARK-4516 |
Netty off-heap memory use causes executors to be killed by OS |
Critical |
Shuffle |
Hector Yee |
Aaron Davidson |
| SPARK-4513 |
Support relational operator '<=>' in Spark SQL |
Major |
SQL |
Ravindra Pesala |
|
| SPARK-4509 |
Revert EC2 tag-based cluster membership patch in branch-1.2 |
Blocker |
EC2 |
Josh Rosen |
Xiangrui Meng |
| SPARK-4506 |
Update documentation to clarify whether standalone-cluster mode is now officially supported |
Major |
Documentation |
Josh Rosen |
Andrew Or |
| SPARK-4498 |
Standalone Master can fail to recognize completed/failed applications |
Blocker |
Deploy, Spark Core |
Harry Brundage |
Mark Hamstra |
| SPARK-4495 |
Memory leak in JobProgressListener due to spark.ui.retainedJobs not being used |
Blocker |
Spark Core |
Josh Rosen |
Josh Rosen |
| SPARK-4487 |
Fix attribute reference resolution error when using ORDER BY. |
Critical |
SQL |
Kousuke Saruta |
|
| SPARK-4484 |
[CORE] Treat maxResultSize as unlimited when set to 0; improve error message |
Critical |
Spark Core |
Nishkam Ravi |
Nishkam Ravi |
| SPARK-4482 |
ReceivedBlockTracker's write ahead log is enabled by default |
Blocker |
Streaming |
Tathagata Das |
Tathagata Das |
| SPARK-4480 |
Avoid many small spills in external data structures |
Blocker |
Spark Core |
Andrew Or |
Andrew Or |
| SPARK-4479 |
Avoid unnecessary defensive copies when Sort based shuffle is on |
Blocker |
SQL |
Michael Armbrust |
Cheng Lian |
| SPARK-4478 |
totalRegisteredExecutors not updated properly |
Major |
Spark Core |
Akshat Aranya |
Akshat Aranya |
| SPARK-4471 |
blockManagerIdFromJson function throws exception while BlockManagerId be null in MetadataFetchFailedException |
Minor |
Spark Core |
SuYan |
SuYan |
| SPARK-4468 |
Wrong Parquet filters are created for all inequality predicates with literals on the left hand side |
Blocker |
SQL |
Cheng Lian |
|
| SPARK-4467 |
Number of elements read is never reset in ExternalSorter |
Blocker |
Shuffle, Spark Core |
Andrew Or |
Tianshuo Deng |
| SPARK-4455 |
Exclude dependency on hbase-annotations module |
Major |
. |
Ted Yu |
Ted Yu |
| SPARK-4448 |
Support ConstantObjectInspector for unwrapping data |
Minor |
SQL |
Cheng Hao |
|
| SPARK-4446 |
MetadataCleaner schedule task with a wrong param for delay time . |
Major |
Spark Core |
Leo |
Leo |
| SPARK-4445 |
Don't display storage level in toDebugString unless RDD is persisted |
Blocker |
Spark Core |
Patrick Wendell |
Prashant Sharma |
| SPARK-4443 |
Statistics bug for external table in spark sql hive |
Critical |
SQL |
Fei Wang |
|
| SPARK-4441 |
Close Tachyon client when TachyonBlockManager is shut down |
Major |
Block Manager |
shimingfei |
shimingfei |
| SPARK-4434 |
spark-submit cluster deploy mode JAR URLs are broken in 1.1.1 |
Blocker |
Deploy, Spark Core |
Josh Rosen |
Andrew Or |
| SPARK-4433 |
Racing condition in zipWithIndex |
Major |
Spark Core |
Xiangrui Meng |
Xiangrui Meng |
| SPARK-4432 |
Resource(InStream) is not closed in TachyonStore |
Major |
Block Manager |
shimingfei |
shimingfei |
| SPARK-4429 |
Build for Scala 2.11 using sbt fails. |
Major |
Build |
Takuya Ueshin |
Takuya Ueshin |
| SPARK-4426 |
The symbol of BitwiseOr is wrong, should not be '&' |
Minor |
SQL |
Kousuke Saruta |
Kousuke Saruta |
| SPARK-4425 |
Handle NaN or Infinity cast to Timestamp correctly |
Major |
SQL |
Takuya Ueshin |
|
| SPARK-4422 |
In some cases, Vectors.fromBreeze get wrong results. |
Minor |
MLlib |
Guoqiang Li |
Guoqiang Li |
| SPARK-4420 |
Change nullability of Cast from DoubleType/FloatType to DecimalType. |
Major |
SQL |
Takuya Ueshin |
|
| SPARK-4415 |
Driver did not exit after python driver had exited. |
Critical |
PySpark, Spark Core |
Davies Liu |
Davies Liu |
| SPARK-4407 |
Thrift server for 0.13.1 doesn't deserialize complex types properly |
Blocker |
SQL |
Cheng Lian |
|
| SPARK-4404 |
SparkSubmitDriverBootstrapper should stop after its SparkSubmit sub-process ends |
Blocker |
Spark Core |
Tao Wang |
Tao Wang |
| SPARK-4391 |
Parquet Filter pushdown flag should be set with SQLConf |
Blocker |
SQL |
Michael Armbrust |
Michael Armbrust |
| SPARK-4390 |
Bad casts to decimal throw instead of returning null |
Blocker |
SQL |
Michael Armbrust |
Michael Armbrust |
| SPARK-4384 |
Too many open files during sort in pyspark |
Blocker |
PySpark |
Davies Liu |
Davies Liu |
| SPARK-4375 |
Assembly built with Maven is missing most of repl classes |
Blocker |
. |
Sandy Ryza |
Sandy Ryza |
| SPARK-4373 |
MLlib unit tests failed maven test |
Critical |
MLlib |
Xiangrui Meng |
Xiangrui Meng |
| SPARK-4372 |
Make LR and SVM's default parameters consistent in Scala and Python |
Major |
MLlib, PySpark |
Xiangrui Meng |
Xiangrui Meng |
| SPARK-4370 |
Limit cores used by Netty transfer service based on executor size |
Critical |
Spark Core |
Aaron Davidson |
Aaron Davidson |
| SPARK-4369 |
TreeModel.predict does not work with RDD |
Blocker |
MLlib, PySpark |
Davies Liu |
Davies Liu |
| SPARK-4355 |
OnlineSummarizer doesn't merge mean correctly |
Major |
MLlib |
Xiangrui Meng |
Xiangrui Meng |
| SPARK-4348 |
pyspark.mllib.random conflicts with random module |
Blocker |
MLlib, PySpark |
Davies Liu |
Davies Liu |
| SPARK-4333 |
Correctly log number of iterations in RuleExecutor |
Minor |
SQL |
Zhongshuai Pei |
|
| SPARK-4326 |
unidoc is broken on master |
Critical |
Build, Documentation |
Xiangrui Meng |
Xiangrui Meng |
| SPARK-4322 |
Struct fields can't be used as sub-expression of grouping fields |
Blocker |
SQL |
Michael Armbrust |
Cheng Lian |
| SPARK-4318 |
Fix empty sum distinct. |
Major |
SQL |
Takuya Ueshin |
|
| SPARK-4313 |
"Thread Dump" link is broken in yarn-cluster mode |
Minor |
Web UI, YARN |
Shixiong Zhu |
Shixiong Zhu |
| SPARK-4312 |
bash can't die |
Major |
Build |
Kousuke Saruta |
Kousuke Saruta |
| SPARK-4310 |
"Submitted" column in Stage page doesn't sort by time |
Minor |
Web UI |
Shixiong Zhu |
Shixiong Zhu |
| SPARK-4309 |
Date type support missing in HiveThriftServer2 |
Major |
SQL |
Cheng Lian |
|
| SPARK-4308 |
SQL operation state is not properly set when exception is thrown |
Minor |
SQL |
Cheng Lian |
Cheng Lian |
| SPARK-4305 |
yarn-alpha profile won't build due to network/yarn module |
Minor |
. |
Sean Owen |
Sean Owen |
| SPARK-4304 |
sortByKey() will fail on empty RDD |
Blocker |
PySpark |
Davies Liu |
Davies Liu |
| SPARK-4301 |
StreamingContext should not allow start() to be called after calling stop() |
Major |
Streaming |
Josh Rosen |
Josh Rosen |
| SPARK-4295 |
[External]Exception throws in SparkSinkSuite although all test cases pass |
Minor |
Streaming |
maji2014 |
|
| SPARK-4292 |
incorrect result set in JDBC/ODBC |
Major |
SQL |
Fei Wang |
|
| SPARK-4291 |
Drop "Code" from network module names |
Major |
Build |
Andrew Or |
Andrew Or |
| SPARK-4282 |
Stopping flag in YarnClientSchedulerBackend should be volatile |
Major |
YARN |
Kousuke Saruta |
Kousuke Saruta |
| SPARK-4281 |
Yarn shuffle service jars need to include dependencies |
Blocker |
Build, YARN |
Andrew Or |
Andrew Or |
| SPARK-4277 |
Support external shuffle service on Worker |
Major |
Deploy, Spark Core |
Aaron Davidson |
Aaron Davidson |
| SPARK-4274 |
NPE in printing the details of query plan |
Minor |
SQL |
Cheng Hao |
|
| SPARK-4270 |
Fix Cast from DateType to DecimalType. |
Major |
SQL |
Takuya Ueshin |
|
| SPARK-4266 |
Avoid expensive JavaScript for StagePages with huge numbers of tasks |
Blocker |
Web UI |
Kay Ousterhout |
Kay Ousterhout |
| SPARK-4264 |
SQL HashJoin induces "refCnt = 0" error in ShuffleBlockFetcherIterator |
Blocker |
. |
Aaron Davidson |
Aaron Davidson |
| SPARK-4260 |
Httpbroadcast should set connection timeout. |
Major |
Spark Core |
Kousuke Saruta |
|
| SPARK-4256 |
MLLib BinaryClassificationMetricComputers try to divide by zero |
Major |
MLlib |
Andrew Bullen |
Andrew Bullen |
| SPARK-4255 |
Table striping is incorrect on page load |
Minor |
Web UI |
Kay Ousterhout |
Kay Ousterhout |
| SPARK-4254 |
MovieLensALS example fails from including Params in closure |
Trivial |
MLlib |
Joseph K. Bradley |
Joseph K. Bradley |
| SPARK-4253 |
Ignore spark.driver.host in yarn-cluster and standalone-cluster mode |
Minor |
YARN |
Tao Wang |
Tao Wang |
| SPARK-4250 |
Create constant null value for Hive Inspectors |
Major |
SQL |
Cheng Hao |
|
| SPARK-4249 |
A problem of EdgePartitionBuilder in Graphx |
Minor |
GraphX |
Cookies |
|
| SPARK-4248 |
[SQL] spark sql not support add jar |
Major |
SQL |
qiaohaijun |
|
| SPARK-4245 |
Fix containsNull of the result ArrayType of CreateArray expression. |
Major |
SQL |
Takuya Ueshin |
Takuya Ueshin |
| SPARK-4244 |
ConstantFolding has to be done before initialize the Generic UDF |
Minor |
SQL |
Cheng Hao |
|
| SPARK-4242 |
Add SASL to external shuffle service |
Major |
Spark Core |
Aaron Davidson |
Aaron Davidson |
| SPARK-4236 |
External shuffle service must cleanup its shuffle files |
Critical |
Spark Core |
Aaron Davidson |
Aaron Davidson |
| SPARK-4230 |
Doc for spark.default.parallelism is incorrect |
Major |
Spark Core |
Sandy Ryza |
Sandy Ryza |
| SPARK-4225 |
jdbc/odbc error when using maven build spark |
Blocker |
Build, SQL |
Fei Wang |
Cheng Lian |
| SPARK-4222 |
FixedLengthBinaryRecordReader should readFully |
Minor |
. |
Jascha Swisher |
Jascha Swisher |
| SPARK-4213 |
SparkSQL - ParquetFilters - No support for LT, LTE, GT, GTE operators |
Blocker |
SQL |
Terry Siu |
|
| SPARK-4207 |
Query which has syntax like 'not like' is not working in Spark SQL |
Major |
SQL |
Ravindra Pesala |
Ravindra Pesala |
| SPARK-4204 |
Utils.exceptionString only return the information for the outermost exception |
Minor |
Spark Core |
Shixiong Zhu |
Shixiong Zhu |
| SPARK-4203 |
Partition directories in random order when inserting into hive table |
Major |
SQL |
Matthew Taylor |
|
| SPARK-4201 |
Can't use concat() on partition column in where condition (Hive compatibility problem) |
Minor |
SQL |
dongxu |
|
| SPARK-4196 |
Streaming + checkpointing + saveAsNewAPIHadoopFiles = NotSerializableException for Hadoop Configuration |
Major |
Streaming |
Sean Owen |
Tathagata Das |
| SPARK-4193 |
Disable doclint in Java 8 to prevent from build error. |
Major |
Build |
Takuya Ueshin |
Takuya Ueshin |
| SPARK-4185 |
JSON schema inference failed when dealing with type conflicts in arrays |
Major |
SQL |
Yin Huai |
Yin Huai |
| SPARK-4183 |
Enable Netty-based BlockTransferService by default |
Major |
. |
Aaron Davidson |
Aaron Davidson |
| SPARK-4182 |
Caching tables containing boolean, binary, array, struct and/or map columns doesn't work |
Blocker |
SQL |
Cheng Lian |
Cheng Lian |
| SPARK-4180 |
SparkContext constructor should throw exception if another SparkContext is already running |
Blocker |
Spark Core |
Josh Rosen |
Josh Rosen |
| SPARK-4177 |
update build doc for already supporting hive 13 in jdbc/cli |
Major |
Documentation |
Fei Wang |
Fei Wang |
| SPARK-4175 |
Exception on stage page |
Critical |
. |
Sandy Ryza |
|
| SPARK-4173 |
EdgePartitionBuilder uses wrong value for first clustered index |
Major |
GraphX |
Ankur Dave |
Ankur Dave |
| SPARK-4169 |
[Core] Locale dependent code |
Major |
Spark Core |
Niklas Wilcke |
Niklas Wilcke |
| SPARK-4158 |
Spark throws exception when Mesos resources are missing |
Major |
Mesos |
Brenden Matthews |
Brenden Matthews |
| SPARK-4155 |
Consolidate usages of <driver> |
Minor |
Spark Core |
Andrew Or |
Andrew Or |
| SPARK-4154 |
Query does not work if it has "not between " in Spark SQL and HQL |
Major |
SQL |
Ravindra Pesala |
Ravindra Pesala |
| SPARK-4153 |
All columns of the application table in the history page are sorted in alphabetical order |
Trivial |
Web UI |
Shixiong Zhu |
Shixiong Zhu |
| SPARK-4152 |
Avoid data change in CTAS while table already existed |
Minor |
SQL |
Cheng Hao |
Cheng Hao |
| SPARK-4148 |
PySpark's sample uses the same seed for all partitions |
Major |
PySpark |
Xiangrui Meng |
Xiangrui Meng |
| SPARK-4146 |
[GraphX] Modify option name according to example doc in SynthBenchmark |
Minor |
GraphX |
Jie Huang |
Jie Huang |
| SPARK-4142 |
Bad Default for GraphLoader Edge Partitions |
Major |
GraphX |
Joseph E. Gonzalez |
Joseph E. Gonzalez |
| SPARK-4138 |
Guard against incompatible settings on the number of executors |
Major |
YARN |
Andrew Or |
Andrew Or |
| SPARK-4137 |
Relative paths don't get handled correctly by spark-ec2 |
Minor |
EC2 |
Nicholas Chammas |
Nicholas Chammas |
| SPARK-4130 |
loadLibSVMFile does not handle extra whitespace |
Major |
MLlib |
Joseph E. Gonzalez |
Joseph E. Gonzalez |
| SPARK-4124 |
Simplify serialization and call API in MLlib Python |
Major |
MLlib, PySpark |
Davies Liu |
Davies Liu |
| SPARK-4121 |
Master build failures after shading commons-math3 |
Blocker |
Build, MLlib, Spark Core |
Xiangrui Meng |
Xiangrui Meng |
| SPARK-4120 |
Join of multiple tables with syntax like SELECT .. FROM T1,T2,T3.. does not work in SparkSQL |
Major |
SQL |
Ravindra Pesala |
Ravindra Pesala |
| SPARK-4113 |
Pyhon UDF on ArrayType |
Blocker |
PySpark, SQL |
Davies Liu |
Davies Liu |
| SPARK-4110 |
Wrong comments about default settings in spark-daemon.sh |
Minor |
Spark Core |
Kousuke Saruta |
Kousuke Saruta |
| SPARK-4107 |
Incorrect handling of Channel.read()'s return value may lead to data truncation |
Blocker |
Spark Core |
Josh Rosen |
Josh Rosen |
| SPARK-4097 |
Race condition in org.apache.spark.ComplexFutureAction.cancel |
Minor |
Spark Core |
Shixiong Zhu |
Shixiong Zhu |
| SPARK-4090 |
Memory leak in snappy-java <IP_ADDRESS>/5 |
Blocker |
Spark Core |
Davies Liu |
|
| SPARK-4089 |
The version number of Spark in _config.yaml is wrong. |
Minor |
Documentation |
Kousuke Saruta |
Kousuke Saruta |
| SPARK-4088 |
Python worker should exit after socket is closed by JVM |
Blocker |
PySpark |
Davies Liu |
Davies Liu |
| SPARK-4085 |
Job will fail if a shuffle file that's read locally gets deleted |
Critical |
Spark Core |
Kay Ousterhout |
Reynold Xin |
| SPARK-4080 |
"IOException: unexpected exception type" while deserializing tasks |
Critical |
Spark Core |
Josh Rosen |
Josh Rosen |
| SPARK-4078 |
New FsPermission instance w/o FsPermission.createImmutable in eventlog |
Major |
Spark Core |
Jie Huang |
Jie Huang |
| SPARK-4077 |
A broken string timestamp value can Spark SQL return wrong values for valid string timestamp values |
Major |
SQL |
Yin Huai |
Venkata Ramana G |
| SPARK-4076 |
Parameter expansion in spark-config is wrong |
Minor |
Spark Core |
Kousuke Saruta |
Kousuke Saruta |
| SPARK-4075 |
Jar url validation is not enough for Jar file |
Minor |
Deploy |
Kousuke Saruta |
Kousuke Saruta |
| SPARK-4071 |
Unroll fails silently if BlockManager size is small |
Major |
Spark Core |
Andrew Or |
Andrew Or |
| SPARK-4068 |
NPE in jsonRDD schema inference |
Critical |
SQL |
Michael Armbrust |
Yin Huai |
| SPARK-4065 |
pyspark will not use ipython on Windows |
Minor |
PySpark |
Michael Griffiths |
Michael Griffiths |
| SPARK-4064 |
NioBlockTransferService should deal with empty messages correctly |
Critical |
Spark Core |
Guoqiang Li |
Guoqiang Li |
| SPARK-4061 |
We cannot use EOL character in the operand of LIKE predicate. |
Major |
SQL |
Kousuke Saruta |
|
| SPARK-4055 |
Inconsistent spelling 'MLlib' and 'MLLib' |
Trivial |
Documentation, MLlib |
Kousuke Saruta |
Kousuke Saruta |
| SPARK-4053 |
Block generator throttling in NetworkReceiverSuite is flaky |
Minor |
Streaming |
Tathagata Das |
Tathagata Das |
| SPARK-4052 |
Use scala.collection.Map for pattern matching instead of using Predef.Map (it is scala.collection.immutable.Map) |
Minor |
SQL |
Yin Huai |
Yin Huai |
| SPARK-4050 |
Caching of temporary tables with projects fail when the final query projects fewer columns |
Blocker |
SQL |
Michael Armbrust |
Michael Armbrust |
| SPARK-4042 |
append columns ids and names before broadcast |
Major |
SQL |
Fei Wang |
|
| SPARK-4041 |
convert attributes names in table scan lowercase when compare with relation attributes |
Major |
SQL |
Fei Wang |
|
| SPARK-4037 |
NPE in JDBC server when calling SET |
Blocker |
SQL |
Michael Armbrust |
Cheng Lian |
| SPARK-4035 |
Wrong format specifier in BlockerManager.scala |
Trivial |
Spark Core |
Shixiong Zhu |
|
| SPARK-4031 |
Read broadcast variables on use |
Major |
Block Manager, Spark Core |
Shivaram Venkataraman |
Shivaram Venkataraman |
| SPARK-4023 |
PySpark's stat.Statistics is broken |
Critical |
MLlib, PySpark |
Xiangrui Meng |
Davies Liu |
| SPARK-4022 |
Replace colt dependency (LGPL) with commons-math |
Critical |
MLlib, Spark Core |
Patrick Wendell |
Sean Owen |
| SPARK-4020 |
Failed executor not properly removed if it has not run tasks |
Major |
Spark Core |
Andrew Or |
Andrew Or |
| SPARK-4019 |
Shuffling with more than 2000 reducers may drop all data when partitons are mostly empty or cause deserialization errors if at least one partition is empty |
Blocker |
Spark Core |
Xiangrui Meng |
Josh Rosen |
| SPARK-4015 |
Documentation in the streaming context references non-existent function |
Trivial |
Streaming |
holdenk |
|
| SPARK-4013 |
Do not run multiple actor systems on each executor |
Major |
Spark Core |
Andrew Or |
Andrew Or |
| SPARK-4010 |
Spark UI returns 500 in yarn-client mode |
Blocker |
Web UI |
Guoqiang Li |
Guoqiang Li |
| SPARK-4006 |
Spark Driver crashes whenever an Executor is registered twice |
Critical |
Block Manager, Spark Core |
Tal Sliwowicz |
Tal Sliwowicz |
| SPARK-4003 |
Add {Big Decimal, Timestamp, Date} types to Java SqlContext |
Major |
SQL |
Adrian Wang |
Adrian Wang |
| SPARK-3997 |
scalastyle should output the error location |
Major |
Build |
Guoqiang Li |
|
| SPARK-3995 |
[PYSPARK] PySpark's sample methods do not work with NumPy 1.9 |
Critical |
PySpark, Spark Core |
Jeremy Freeman |
Jeremy Freeman |
| SPARK-3994 |
countByKey / countByValue do not go through Aggregator |
Major |
Spark Core |
Aaron Davidson |
Aaron Davidson |
| SPARK-3993 |
python worker may hang after reused from take() |
Blocker |
PySpark |
Davies Liu |
Davies Liu |
| SPARK-3990 |
kryo.KryoException caused by ALS.trainImplicit in pyspark |
Critical |
MLlib, PySpark |
Gen TANG |
Xiangrui Meng |
| SPARK-3987 |
NNLS generates incorrect result |
Major |
MLlib |
Debasish Das |
Shuo Xiang |
| SPARK-3986 |
Fix package names to fit their directory names. |
Minor |
SQL |
Takuya Ueshin |
Takuya Ueshin |
| SPARK-3985 |
json file path is not right |
Minor |
Examples, SQL |
Adrian Wang |
Adrian Wang |
| SPARK-3983 |
Scheduler delay (shown in the UI) is incorrect |
Major |
Web UI |
Kay Ousterhout |
Kay Ousterhout |
| SPARK-3979 |
Yarn backend's default file replication should match HDFS's default one |
Minor |
YARN |
Marcelo Vanzin |
Marcelo Vanzin |
| SPARK-3973 |
Print callSite information for broadcast variables |
Minor |
Spark Core |
Shivaram Venkataraman |
Shivaram Venkataraman |
| SPARK-3971 |
Failed to deserialize Vector in cluster mode |
Blocker |
MLlib, PySpark |
Davies Liu |
Davies Liu |
| SPARK-3970 |
Remove duplicate removal of local dirs |
Major |
. |
Liang-Chi Hsieh |
Liang-Chi Hsieh |
| SPARK-3967 |
Spark applications fail in yarn-cluster mode when the directories configured in yarn.nodemanager.local-dirs are located on different disks/partitions |
Major |
YARN |
Christophe Préaud |
Christophe Préaud |
| SPARK-3966 |
Fix nullabilities of Cast related to DateType. |
Major |
SQL |
Takuya Ueshin |
Takuya Ueshin |
| SPARK-3962 |
Mark spark dependency as "provided" in external libraries |
Blocker |
Streaming |
Patrick Wendell |
Prashant Sharma |
| SPARK-3960 |
We can apply unary minus only to literal. |
Critical |
SQL |
Kousuke Saruta |
Kousuke Saruta |
| SPARK-3959 |
SqlParser fails to parse literal -9223372036854775808 (Long.MinValue). |
Critical |
SQL |
Kousuke Saruta |
|
| SPARK-3952 |
Python examples in Streaming Programming Guide |
Blocker |
PySpark, Streaming |
Davies Liu |
|
| SPARK-3948 |
Sort-based shuffle can lead to assorted stream-corruption exceptions |
Blocker |
Shuffle |
Saisai Shao |
Saisai Shao |
| SPARK-3946 |
.gitignore in /python includes wrong directory. |
Minor |
Build |
Masayoshi TSUZUKI |
Masayoshi TSUZUKI |
| SPARK-3945 |
Write properties of hive-site.xml to HiveContext when initilize session state In SparkSQLEnv.scala |
Major |
SQL |
Gankun Luo |
Gankun Luo |
| SPARK-3941 |
_remainingMem should not increase twice when updateBlockInfo |
Major |
Spark Core |
Zhang, Liye |
Zhang, Liye |
| SPARK-3940 |
SQL console prints error messages three times |
Major |
SQL |
XiaoJing wang |
XiaoJing wang |
| SPARK-3936 |
Remove auto join elimination and introduce aggregateMessages |
Blocker |
GraphX |
Pedro Rodriguez |
Ankur Dave |
| SPARK-3935 |
Unused variable in PairRDDFunctions.scala |
Minor |
Spark Core |
Fei Wang |
Fei Wang |
| SPARK-3934 |
RandomForest bug in sanity check in DTStatsAggregator |
Major |
MLlib |
Joseph K. Bradley |
Joseph K. Bradley |
| SPARK-3925 |
Do not consider the ordering of qualifiers during comparison |
Major |
SQL |
Liang-Chi Hsieh |
|
| SPARK-3923 |
All Standalone Mode services time out with each other |
Blocker |
Deploy |
Aaron Davidson |
Aaron Davidson |
| SPARK-3921 |
WorkerWatcher in Standalone mode fail to come up due to invalid workerUrl |
Critical |
. |
Aaron Davidson |
Aaron Davidson |
| SPARK-3918 |
Forget Unpersist in RandomForest.scala(train Method) |
Major |
MLlib |
junlong |
Joseph K. Bradley |
| SPARK-3912 |
FlumeStreamSuite is flaky, fails either with port binding issues or data not being reliably sent |
Critical |
Streaming |
Tathagata Das |
Tathagata Das |
| SPARK-3909 |
A corrupted format in Sphinx documents and building warnings |
Minor |
PySpark |
Tomohiko K. |
|
| SPARK-3905 |
The keys for sorting the columns of Executor page ,Stage page Storage page are incorrect |
Major |
Web UI |
Guoqiang Li |
Guoqiang Li |
| SPARK-3904 |
HQL doesn't support the ConstantObjectInspector to pass into UDFs |
Major |
SQL |
Cheng Hao |
Cheng Hao |
| SPARK-3900 |
ApplicationMaster's shutdown hook fails and IllegalStateException is thrown. |
Critical |
YARN |
Kousuke Saruta |
|
| SPARK-3899 |
wrong links in streaming doc |
Major |
Documentation |
Fei Wang |
|
| SPARK-3889 |
JVM dies with SIGBUS, resulting in ConnectionManager failed ACK |
Critical |
Spark Core |
Aaron Davidson |
Aaron Davidson |
| SPARK-3877 |
The exit code of spark-submit is still 0 when an yarn application fails |
Major |
YARN |
Shixiong Zhu |
Shixiong Zhu |
| SPARK-3869 |
./bin/spark-class miss Java version with _JAVA_OPTIONS set |
Major |
Spark Shell |
Tomohiko K. |
|
| SPARK-3858 |
SchemaRDD.generate ignores alias argument |
Minor |
SQL |
Nathan Howell |
|
| SPARK-3855 |
Binding Exception when running PythonUDFs |
Major |
PySpark, SQL |
Michael Armbrust |
Michael Armbrust |
| SPARK-3853 |
JsonRDD does not support converting fields to type Timestamp |
Major |
SQL |
Michael Timper |
|
| SPARK-3848 |
yarn alpha doesn't build on master |
Blocker |
YARN |
Thomas Graves |
Kousuke Saruta |
| SPARK-3843 |
Cleanup scalastyle.txt at the end of running dev/scalastyle |
Trivial |
Build |
Kousuke Saruta |
Kousuke Saruta |
| SPARK-3840 |
Spark EC2 templates fail when variables are missing |
Major |
EC2 |
Allan Douglas R. de Oliveira |
|
| SPARK-3834 |
Backticks not correctly handled in subquery aliases |
Blocker |
SQL |
Michael Armbrust |
Ravindra Pesala |
| SPARK-3829 |
Make Spark logo image on the header of HistoryPage as a link to HistoryPage's page #1 |
Minor |
Web UI |
Kousuke Saruta |
Kousuke Saruta |
| SPARK-3827 |
Very long RDD names are not rendered properly in web UI |
Minor |
Web UI |
Hossein Falaki |
|
| SPARK-3825 |
Log more information when unrolling a block fails |
Major |
Block Manager, Spark Core |
Andrew Or |
Andrew Or |
| SPARK-3824 |
Spark SQL should cache in MEMORY_AND_DISK by default |
Blocker |
SQL |
Patrick Wendell |
Cheng Lian |
| SPARK-3816 |
Add configureOutputJobPropertiesForStorageHandler to JobConf in SparkHadoopWriter |
Major |
SQL |
Alex Liu |
|
| SPARK-3814 |
Support for Bitwise AND(&), OR(|) ,XOR(^), NOT(~) in Spark HQL and SQL |
Minor |
SQL |
Yana Kadiyska |
Ravindra Pesala |
| SPARK-3810 |
Rule PreInsertionCasts doesn't handle partitioned table properly |
Minor |
SQL |
Cheng Lian |
|
| SPARK-3809 |
make HiveThriftServer2Suite work correctly |
Major |
SQL |
Fei Wang |
|
| SPARK-3808 |
PySpark fails to start in Windows |
Blocker |
PySpark, Windows |
Masayoshi TSUZUKI |
Masayoshi TSUZUKI |
| SPARK-3807 |
SparkSql does not work for tables created using custom serde |
Major |
SQL |
chirag aggarwal |
|
| SPARK-3806 |
minor bug in CliSuite |
Major |
SQL |
Fei Wang |
|
| SPARK-3803 |
ArrayIndexOutOfBoundsException found in executing computePrincipalComponents |
Major |
MLlib |
Masaru Dobashi |
Sean Owen |
| SPARK-3800 |
BindingException when grouping on nested fields |
Critical |
SQL |
Michael Armbrust |
Michael Armbrust |
| SPARK-3798 |
Corrupted projection in Generator |
Blocker |
SQL |
Michael Armbrust |
Michael Armbrust |
| SPARK-3794 |
Building spark core fails due to inadvertent dependency on Commons IO |
Major |
Spark Core |
Tomohiko K. |
|
| SPARK-3791 |
HiveThriftServer2 returns 0.12.0 to ODBC SQLGetInfo call |
Major |
SQL |
Cheng Lian |
Cheng Lian |
| SPARK-3788 |
Yarn dist cache code is not friendly to HDFS HA, Federation |
Major |
YARN |
Marcelo Vanzin |
Marcelo Vanzin |
| SPARK-3783 |
The type parameters for SparkContext.accumulable are inconsistent Accumulable itself |
Minor |
Spark Core |
Nathan Kronenfeld |
Nathan Kronenfeld |
| SPARK-3776 |
Wrong conversion to Catalyst for Option[Product] |
Major |
SQL |
Renat Yusupov |
|
| SPARK-3773 |
Sphinx build warnings |
Minor |
PySpark |
Tomohiko K. |
|
| SPARK-3772 |
RDD operation on IPython REPL failed with an illegal port number |
Major |
PySpark |
Tomohiko K. |
|
| SPARK-3771 |
AppendingParquetOutputFormat should use reflection to prevent from breaking binary-compatibility. |
Major |
SQL |
Takuya Ueshin |
|
| SPARK-3763 |
The example of building with sbt should be "sbt assembly" instead of "sbt compile" |
Trivial |
Documentation |
Kousuke Saruta |
Kousuke Saruta |
| SPARK-3762 |
clear all SparkEnv references after stop |
Critical |
Spark Core |
Davies Liu |
Davies Liu |
| SPARK-3759 |
SparkSubmitDriverBootstrapper should return exit code of driver process |
Minor |
Deploy |
Eric Eijkelenboom |
Eric Eijkelenboom |
| SPARK-3757 |
mvn clean doesn't delete some files |
Trivial |
Build |
Masayoshi TSUZUKI |
|
| SPARK-3756 |
Include possible MultiException when detecting port collisions |
Major |
. |
Fei Wang |
Fei Wang |
| SPARK-3755 |
Do not bind port 1 - 1024 to server in spark |
Major |
Spark Core |
Fei Wang |
Fei Wang |
| SPARK-3749 |
Bugs in broadcast of large RDD |
Critical |
PySpark |
Davies Liu |
Davies Liu |
| SPARK-3747 |
TaskResultGetter could incorrectly abort a stage if it cannot get result for a specific task |
Major |
Spark Core |
Reynold Xin |
Reynold Xin |
| SPARK-3746 |
Failure to lock hive client when creating tables |
Major |
. |
Michael Armbrust |
Michael Armbrust |
| SPARK-3745 |
curl on maven search repo (apache rat) url returns search status, not jar file |
Major |
Build |
shane knapp |
|
| SPARK-3744 |
FlumeStreamSuite will fail during port contention |
Blocker |
. |
Patrick Wendell |
Tathagata Das |
| SPARK-3741 |
ConnectionManager.sendMessage may not propagate errors to MessageStatus |
Minor |
Spark Core |
Shixiong Zhu |
|
| SPARK-3736 |
Workers should reconnect to Master if disconnected |
Critical |
Spark Core |
Andrew Ash |
Matthew Cheah |
| SPARK-3734 |
DriverRunner should not read SPARK_HOME from submitter's environment |
Major |
Deploy |
Josh Rosen |
Josh Rosen |
| SPARK-3731 |
RDD caching stops working in pyspark after some time |
Critical |
PySpark, Spark Core |
Milan Straka |
Davies Liu |
| SPARK-3729 |
Null-pointer when constructing a HiveContext when settings are present |
Blocker |
SQL |
Michael Armbrust |
Michael Armbrust |
| SPARK-3721 |
Broadcast Variables above 2GB break in PySpark |
Major |
PySpark |
Brad Miller |
Davies Liu |
| SPARK-3710 |
YARN integration test is flaky |
Blocker |
YARN |
Patrick Wendell |
Marcelo Vanzin |
| SPARK-3709 |
Executors don't always report broadcast block removal properly back to the driver |
Blocker |
Spark Core |
Patrick Wendell |
Reynold Xin |
| SPARK-3708 |
Backticks aren't handled correctly in aliases |
Major |
SQL |
Michael Armbrust |
Ravindra Pesala |
| SPARK-3707 |
Type Coercion for DIV doesn't work for non-numeric argument |
Minor |
SQL |
Cheng Hao |
|
| SPARK-3706 |
Cannot run IPython REPL with IPYTHON set to "1" and PYSPARK_PYTHON unset |
Major |
PySpark |
Tomohiko K. |
|
| SPARK-3705 |
add case for VoidObjectInspector in inspectorToDataType |
Major |
SQL |
Fei Wang |
|
| SPARK-3704 |
Short (TINYINT) incorrectly handled in thrift JDBC/ODBC server |
Major |
SQL |
Fei Wang |
|
| SPARK-3701 |
Some clean-up work after the refactoring of MLlib's SerDe for PySpark |
Minor |
MLlib, PySpark |
Xiangrui Meng |
Xiangrui Meng |
| SPARK-3699 |
sbt console tasks don't clean up SparkContext |
Minor |
SQL |
William Benton |
William Benton |
| SPARK-3697 |
HistoryServer cann't list event Log when there was a no permissions directory in the $spark.eventLog.dir |
Minor |
Spark Core |
yangping wu |
|
| SPARK-3695 |
Enable to show host and port in block fetch failure |
Minor |
Input/Output |
Adrian Wang |
Adrian Wang |
| SPARK-3690 |
Closing shuffle writers we swallow more important exception |
Minor |
Spark Core |
Egor Pakhomov |
|
| SPARK-3686 |
flume.SparkSinkSuite.Success is flaky |
Blocker |
Streaming |
Patrick Wendell |
Hari Shreedharan |
| SPARK-3681 |
Failed to serialized ArrayType or MapType after accessing them in Python |
Major |
PySpark |
Davies Liu |
Davies Liu |
| SPARK-3680 |
java.lang.Exception: makeCopy when using HiveGeneric UDFs on Converted Parquet Metastore tables |
Critical |
SQL |
Michael Armbrust |
Michael Armbrust |
| SPARK-3679 |
pickle the exact globals of functions |
Critical |
PySpark |
Davies Liu |
Davies Liu |
| SPARK-3676 |
jdk version lead to spark sql test suite error |
Major |
SQL |
Fei Wang |
|
| SPARK-3661 |
spark.*.memory is ignored in cluster mode |
Critical |
Spark Core |
Andrew Or |
Andrew Or |
| SPARK-3659 |
Set EC2 version to 1.1.0 in master branch |
Minor |
EC2 |
Shivaram Venkataraman |
Shivaram Venkataraman |
| SPARK-3657 |
yarn alpha YarnRMClientImpl throws NPE appMasterRequest.setTrackingUrl starting spark-shell |
Blocker |
YARN |
Thomas Graves |
Kousuke Saruta |
| SPARK-3654 |
Implement all extended HiveQL statements/commands with a separate parser combinator |
Major |
SQL |
Cheng Lian |
Ravindra Pesala |
| SPARK-3649 |
ClassCastException in GraphX custom serializers when sort-based shuffle spills |
Major |
GraphX |
Ankur Dave |
Ankur Dave |
| SPARK-3647 |
Shaded Guava patch causes access issues with package private classes |
Critical |
Spark Core |
Marcelo Vanzin |
Marcelo Vanzin |
| SPARK-3645 |
Make caching using SQL commands eager by default, with the option of being lazy |
Critical |
SQL |
Michael Armbrust |
Cheng Lian |
| SPARK-3641 |
Correctly populate SparkPlan.currentContext |
Critical |
SQL |
Yin Huai |
Michael Armbrust |
| SPARK-3639 |
Kinesis examples set master as local |
Minor |
Examples, Streaming |
Aniket Bhatnagar |
Aniket Bhatnagar |
| SPARK-3638 |
Commons HTTP client dependency conflict in extras/kinesis-asl module |
Major |
Examples, Streaming |
Aniket Bhatnagar |
|
| SPARK-3635 |
Find Strongly Connected Components with Graphx has a small bug |
Trivial |
GraphX |
Oded Zimerman |
|
| SPARK-3633 |
Fetches failure observed after SPARK-2711 |
Blocker |
Block Manager |
Nishkam Ravi |
|
| SPARK-3632 |
ConnectionManager can run out of receive threads with authentication on |
Critical |
Spark Core |
Thomas Graves |
Thomas Graves |
| SPARK-3628 |
Don't apply accumulator updates multiple times for tasks in result stages |
Blocker |
Spark Core |
Matei Zaharia |
Nan Zhu |
| SPARK-3627 |
spark on yarn reports success even though job fails |
Critical |
YARN |
Thomas Graves |
Thomas Graves |
| SPARK-3618 |
Store analyzed plans for temp tables |
Major |
SQL |
Michael Armbrust |
Michael Armbrust |
| SPARK-3615 |
Kafka test should not hard code Zookeeper port |
Blocker |
Streaming |
Patrick Wendell |
Saisai Shao |
| SPARK-3612 |
Executor shouldn't quit if heartbeat message fails to reach the driver |
Major |
Spark Core |
Reynold Xin |
Sandy Ryza |
| SPARK-3610 |
History server log name should not be based on user input |
Critical |
Spark Core |
SK |
Kousuke Saruta |
| SPARK-3609 |
Add sizeInBytes statistics to Limit operator |
Major |
SQL |
Cheng Lian |
Cheng Lian |
| SPARK-3608 |
Spark EC2 Script does not correctly break when AWS tagging succeeds. |
Critical |
EC2 |
Vida Ha |
Vida Ha |
| SPARK-3606 |
Spark-on-Yarn AmIpFilter does not work with Yarn HA. |
Major |
YARN |
Marcelo Vanzin |
Marcelo Vanzin |
| SPARK-3605 |
Typo in SchemaRDD JavaDoc |
Trivial |
SQL |
Sandy Ryza |
|
| SPARK-3598 |
cast to timestamp should be the same as hive |
Major |
SQL |
Adrian Wang |
Adrian Wang |
| SPARK-3597 |
MesosSchedulerBackend does not implement killTask |
Major |
Mesos |
Brenden Matthews |
Brenden Matthews |
| SPARK-3592 |
applySchema to an RDD of Row |
Critical |
PySpark, SQL |
Davies Liu |
Davies Liu |
| SPARK-3584 |
sbin/slaves doesn't work when we use password authentication for SSH |
Major |
Spark Core |
Kousuke Saruta |
Kousuke Saruta |
| SPARK-3582 |
Spark SQL having issue with existing Hive UDFs which take Map as a parameter |
Major |
SQL |
Saurabh Santhosh |
Adrian Wang |
| SPARK-3579 |
Jekyll doc generation is different across environments |
Major |
Documentation |
Patrick Wendell |
Patrick Wendell |
| SPARK-3578 |
GraphGenerators.sampleLogNormal sometimes returns too-large result |
Minor |
GraphX |
Ankur Dave |
Ankur Dave |
| SPARK-3571 |
Spark standalone cluster mode doesn't work. |
Blocker |
Spark Core |
Kousuke Saruta |
Kousuke Saruta |
| SPARK-3567 |
appId field in SparkDeploySchedulerBackend should be volatile |
Major |
Spark Core |
Kousuke Saruta |
Kousuke Saruta |
| SPARK-3560 |
In yarn-cluster mode, the same jars are distributed through multiple mechanisms. |
Critical |
YARN |
Sandy Ryza |
Min Shen |
| SPARK-3559 |
appendReadColumnIDs and appendReadColumnNames introduce unnecessary columns in the lists of needed column ids and column names stored in hiveConf |
Blocker |
SQL |
Yin Huai |
Venkata Ramana G |
| SPARK-3555 |
UI port contention suite flakey |
Blocker |
Spark Core |
Patrick Wendell |
Andrew Or |
| SPARK-3546 |
InputStream of ManagedBuffer is not closed and causes running out of file descriptor |
Critical |
Spark Core |
Kousuke Saruta |
Kousuke Saruta |
| SPARK-3543 |
Write TaskContext in Java and expose it through a static accessor |
Critical |
Spark Core |
Patrick Wendell |
Prashant Sharma |
| SPARK-3539 |
Task description "apply at Option.scala:120"; no user code involved |
Minor |
Spark Core |
John Salvatier |
|
| SPARK-3536 |
SELECT on empty parquet table throws exception |
Major |
SQL |
Michael Armbrust |
Ravindra Pesala |
| SPARK-3535 |
Spark on Mesos not correctly setting heap overhead |
Major |
Mesos |
Brenden Matthews |
Brenden Matthews |
| SPARK-3534 |
Avoid running MLlib and Streaming tests when testing SQL PRs |
Blocker |
Project Infra, SQL |
Michael Armbrust |
Nicholas Chammas |
| SPARK-3531 |
select null from table would throw a MatchError |
Major |
SQL |
Adrian Wang |
|
| SPARK-3527 |
Strip the physical plan message margin |
Trivial |
SQL |
Cheng Hao |
Cheng Hao |
| SPARK-3519 |
PySpark RDDs are missing the distinct(n) method |
Major |
PySpark, Spark Core |
Nicholas Chammas |
Matthew Farrellee |
| SPARK-3518 |
Remove useless statement in JsonProtocol |
Minor |
Spark Core |
Kousuke Saruta |
Kousuke Saruta |
| SPARK-3515 |
ParquetMetastoreSuite fails when executed together with other suites under Maven |
Major |
SQL |
Cheng Lian |
Cheng Lian |
| SPARK-3501 |
Hive SimpleUDF will create duplicated type cast which cause exception in constant folding |
Minor |
SQL |
Cheng Hao |
Cheng Hao |
| SPARK-3500 |
coalesce() and repartition() of SchemaRDD is broken |
Critical |
PySpark, SQL |
Davies Liu |
Davies Liu |
| SPARK-3496 |
Block replication can by mistake choose driver BlockManager as a peer for replication |
Critical |
Block Manager, Spark Core, Streaming |
Tathagata Das |
Tathagata Das |
| SPARK-3495 |
Block replication fails continuously when the replication target node is dead |
Critical |
Block Manager, Spark Core, Streaming |
Tathagata Das |
Tathagata Das |
| SPARK-3494 |
DecisionTree overflow error in calculating maxMemoryUsage |
Major |
MLlib |
Joseph K. Bradley |
Joseph K. Bradley |
| SPARK-3490 |
Alleviate port collisions during tests |
Major |
Spark Core |
Andrew Or |
Andrew Or |
| SPARK-3465 |
Task metrics are not aggregated correctly in local mode |
Blocker |
Spark Core |
Davies Liu |
Davies Liu |
| SPARK-3456 |
YarnAllocator can lose container requests to RM |
Critical |
YARN |
Thomas Graves |
Thomas Graves |
| SPARK-3452 |
Maven build should skip publishing artifacts people shouldn't depend on |
Critical |
Build |
Patrick Wendell |
Prashant Sharma |
| SPARK-3448 |
SpecificMutableRow.update doesn't check for null |
Minor |
SQL |
Cheng Lian |
Cheng Lian |
| SPARK-3447 |
Kryo NPE when serializing JListWrapper |
Major |
SQL |
Michael Armbrust |
Michael Armbrust |
| SPARK-3433 |
Mima false-positives with @DeveloperAPI and @Experimental annotations |
Minor |
Build |
Josh Rosen |
Prashant Sharma |
| SPARK-3429 |
Don't include the empty string "" as a defaultAclUser |
Major |
Spark Core |
Andrew Ash |
Andrew Ash |
| SPARK-3426 |
Sort-based shuffle compression behavior is inconsistent |
Blocker |
Spark Core |
Andrew Or |
Josh Rosen |
| SPARK-3417 |
Use of old-style classes in pyspark |
Minor |
. |
Matthew Rocklin |
|
| SPARK-3415 |
Using sys.stderr in pyspark results in error |
Major |
PySpark |
Ward Viaene |
|
| SPARK-3414 |
Case insensitivity breaks when unresolved relation contains attributes with uppercase letters in their names |
Critical |
SQL |
Cheng Lian |
Michael Armbrust |
| SPARK-3412 |
Add Missing Types for Row API |
Minor |
SQL |
Cheng Hao |
Cheng Hao |
| SPARK-3408 |
Limit operator doesn't work with sort based shuffle |
Major |
SQL |
Reynold Xin |
Reynold Xin |
| SPARK-3406 |
Python persist API does not have a default storage level |
Minor |
PySpark |
holdenk |
|
| SPARK-3404 |
SparkSubmitSuite fails with "spark-submit exits with code 1" |
Critical |
Build |
Sean Owen |
|
| SPARK-3399 |
Test for PySpark should ignore HADOOP_CONF_DIR and YARN_CONF_DIR |
Major |
PySpark |
Kousuke Saruta |
|
| SPARK-3395 |
[SQL] DSL uses incorrect attribute ids after a distinct() |
Minor |
SQL |
Eric Liang |
Eric Liang |
| SPARK-3394 |
TakeOrdered crashes when limit is 0 |
Minor |
Spark Core, SQL |
Eric Liang |
Eric Liang |
| SPARK-3392 |
Set command always get <undefined> for key "mapred.reduce.tasks" |
Trivial |
SQL |
Cheng Hao |
Cheng Hao |
| SPARK-3390 |
sqlContext.jsonRDD fails on a complex structure of JSON array and JSON object nesting |
Critical |
SQL |
Vida Ha |
Yin Huai |
| SPARK-3378 |
Replace the word "SparkSQL" with right word "Spark SQL" |
Trivial |
Documentation |
Kousuke Saruta |
|
| SPARK-3377 |
Metrics can be accidentally aggregated against our intention |
Critical |
Spark Core |
Kousuke Saruta |
|
| SPARK-3375 |
spark on yarn container allocation issues |
Blocker |
YARN |
Thomas Graves |
Thomas Graves |
| SPARK-3372 |
MLlib doesn't pass maven build / checkstyle due to multi-byte character contained in Gradient.scala |
Blocker |
MLlib |
Kousuke Saruta |
Kousuke Saruta |
| SPARK-3371 |
Spark SQL: Renaming a function expression with group by gives error |
Major |
SQL |
Pei-Lun Lee |
Ravindra Pesala |
| SPARK-3363 |
[SQL] Type Coercion should support every type to have null value |
Major |
SQL |
Adrian Wang |
Adrian Wang |
| SPARK-3362 |
[SQL] bug in CaseWhen resolve |
Major |
SQL |
Adrian Wang |
Adrian Wang |
| SPARK-3358 |
PySpark worker fork()ing performance regression in m3.* / PVM instances |
Major |
PySpark |
Josh Rosen |
|
| SPARK-3349 |
Incorrect partitioning after LIMIT operator |
Major |
SQL |
Eric Liang |
Eric Liang |
| SPARK-3347 |
Spark on yarn alpha doesn't compile due to SPARK-2889 |
Blocker |
YARN |
Thomas Graves |
Marcelo Vanzin |
| SPARK-3345 |
Do correct parameters for ShuffleFileGroup |
Minor |
Spark Core |
Liang-Chi Hsieh |
Liang-Chi Hsieh |
| SPARK-3341 |
The dataType of Sqrt expression should be DoubleType. |
Major |
SQL |
Takuya Ueshin |
Takuya Ueshin |
| SPARK-3339 |
Support for skipping json lines that fail to parse |
Critical |
SQL |
Michael Armbrust |
Yin Huai |
| SPARK-3338 |
Respect user setting of spark.submit.pyFiles |
Major |
Spark Core |
Andrew Or |
Andrew Or |
| SPARK-3335 |
[Spark SQL] In pyspark, cannot use broadcast variables in UDF |
Major |
PySpark, SQL |
kay feng |
Davies Liu |
| SPARK-3332 |
Tagging is not atomic with launching instances on EC2 |
Major |
EC2 |
Allan Douglas R. de Oliveira |
Josh Rosen |
| SPARK-3331 |
PEP8 tests fail because they check unzipped py4j code |
Minor |
Build |
Sean Owen |
|
| SPARK-3329 |
HiveQuerySuite SET tests depend on map orderings |
Trivial |
SQL |
William Benton |
William Benton |
| SPARK-3319 |
Resolve spark.jars, spark.files, and spark.submit.pyFiles etc. |
Major |
Spark Core |
Andrew Or |
Andrew Or |
| SPARK-3308 |
Ability to read JSON Arrays as tables |
Critical |
SQL |
Michael Armbrust |
Yin Huai |
| SPARK-3304 |
ApplicationMaster's Finish status is wrong when uncaught exception is thrown from ReporterThread |
Major |
YARN |
Kousuke Saruta |
Kousuke Saruta |
| SPARK-3303 |
SparkContextSchedulerCreationSuite test failed when mesos native lib is set |
Minor |
Spark Core |
Fei Wang |
Fei Wang |
| SPARK-3302 |
The wrong version information in SparkContext |
Major |
Spark Core |
Guoqiang Li |
Guoqiang Li |
| SPARK-3301 |
The spark version in the welcome message of pyspark is not correct |
Minor |
PySpark |
Guoqiang Li |
Guoqiang Li |
| SPARK-3297 |
[Spark SQL][UI] SchemaRDD toString with many columns messes up Storage tab display |
Minor |
SQL, Web UI |
Evan Chan |
Hossein Falaki |
| SPARK-3286 |
Cannot view ApplicationMaster UI when Yarn’s url scheme is https |
Major |
Web UI, YARN |
Benoy Antony |
|
| SPARK-3279 |
Remove useless field variable in ApplicationMaster |
Major |
YARN |
Kousuke Saruta |
|
| SPARK-3273 |
We should read the version information from the same place. |
Minor |
Spark Shell |
Guoqiang Li |
Guoqiang Li |
| SPARK-3268 |
DoubleType should support modulus |
Minor |
SQL |
Chris Grier |
Venkata Ramana G |
| SPARK-3263 |
PR #720 broke GraphGenerator.logNormal |
Major |
GraphX |
RJ Nowling |
|
| SPARK-3260 |
Yarn - pass acls along with executor launch |
Major |
YARN |
Thomas Graves |
Thomas Graves |
| SPARK-3233 |
Executor never stop its SparnEnv, BlockManager, ConnectionManager etc. |
Critical |
Spark Core |
Kousuke Saruta |
Kousuke Saruta |
| SPARK-3225 |
Typo in script |
Trivial |
Deploy |
Tao Wang |
Tao Wang |
| SPARK-3223 |
runAsSparkUser cannot change HDFS write permission properly in mesos cluster mode |
Critical |
Input/Output, Mesos |
Jongyoul Lee |
Jongyoul Lee |
| SPARK-3217 |
Shaded Guava jar doesn't play well with Maven build when SPARK_PREPEND_CLASSES is set |
Major |
Build |
Cheng Lian |
Marcelo Vanzin |
| SPARK-3212 |
Improve the clarity of caching semantics |
Blocker |
SQL |
Michael Armbrust |
Michael Armbrust |
| SPARK-3211 |
.take() is OOM-prone when there are empty partitions |
Major |
Spark Core |
Andrew Ash |
Andrew Ash |
| SPARK-3206 |
Error in PageRank values |
Major |
GraphX |
Peter Fontana |
Ankur Dave |
| SPARK-3190 |
Creation of large graph(> 2.15 B nodes) seems to be broken:possible overflow somewhere |
Critical |
GraphX |
npanj |
Ankur Dave |
| SPARK-3178 |
setting SPARK_WORKER_MEMORY to a value without a label (m or g) sets the worker memory limit to zero |
Major |
. |
Jon Haddad |
Bill Bejeck |
| SPARK-3177 |
Yarn-alpha ClientBaseSuite Unit test failed |
Minor |
YARN |
Chester |
|
| SPARK-3121 |
Wrong implementation of implicit bytesWritableConverter |
Critical |
Spark Core |
Jakub Dubovsky |
|
| SPARK-3067 |
JobProgressPage could not show Fair Scheduler Pools section sometimes |
Minor |
Spark Core, Web UI |
YanTang Zhai |
YanTang Zhai |
| SPARK-3061 |
Maven build fails in Windows OS |
Minor |
Build |
Masayoshi TSUZUKI |
Josh Rosen |
| SPARK-3032 |
Potential bug when running sort-based shuffle with sorting using TimSort |
Blocker |
Shuffle |
Saisai Shao |
Saisai Shao |
| SPARK-2981 |
PartitionStrategy: VertexID hash overflow |
Major |
GraphX |
Larry Xiao |
|
| SPARK-2955 |
Test code fails to compile with "mvn compile" without "install" |
Minor |
Build |
Sean Owen |
Sean Owen |
| SPARK-2951 |
SerDeUtils.pythonToPairRDD fails on RDDs of pickled array.arrays in Python 2.6 |
Major |
PySpark |
Josh Rosen |
Davies Liu |
| SPARK-2917 |
Avoid CTAS creates table in logical plan analyzing. |
Minor |
SQL |
Cheng Hao |
Cheng Hao |
| SPARK-2915 |
Storage summary table UI glitch when using sparkSQL |
Minor |
Web UI |
Hossein Falaki |
Hossein Falaki |
| SPARK-2890 |
Spark SQL should allow SELECT with duplicated columns |
Major |
SQL |
Jianshi Huang |
Michael Armbrust |
| SPARK-2889 |
Spark creates Hadoop Configuration objects inconsistently |
Minor |
Spark Core |
Marcelo Vanzin |
Marcelo Vanzin |
| SPARK-2881 |
Snappy is now default codec - could lead to conflicts since uses /tmp |
Blocker |
Spark Core |
Thomas Graves |
Patrick Wendell |
| SPARK-2866 |
ORDER BY attributes must appear in SELECT clause |
Critical |
SQL |
Michael Armbrust |
Michael Armbrust |
| SPARK-2781 |
Analyzer should check resolution of LogicalPlans |
Major |
SQL |
Aaron Staple |
Michael Armbrust |
| SPARK-2760 |
Caching tables from multiple databases does not work |
Critical |
SQL |
Michael Armbrust |
Michael Armbrust |
| SPARK-2742 |
The variable inputFormatInfo and inputFormatMap never used |
Minor |
YARN |
meiyoula |
|
| SPARK-2693 |
Support for UDAF Hive Aggregates like PERCENTILE |
Critical |
SQL |
Michael Armbrust |
Ravindra Pesala |
| SPARK-2634 |
MapOutputTrackerWorker.mapStatuses should be thread-safe |
Major |
Spark Core |
Shixiong Zhu |
|
| SPARK-2626 |
Stop SparkContext in all examples |
Major |
Spark Core |
Andrew Or |
Sean Owen |
| SPARK-2548 |
JavaRecoverableWordCount is missing |
Minor |
Documentation, Streaming |
Xiangrui Meng |
|
| SPARK-2546 |
Configuration object thread safety issue |
Critical |
Spark Core |
Andrew Ash |
Josh Rosen |
| SPARK-2482 |
Resolve sbt warnings during build |
Major |
. |
Guoqiang Li |
Guoqiang Li |
| SPARK-2435 |
Add shutdown hook to bin/pyspark |
Major |
PySpark |
Andrew Or |
Matthew Farrellee |
| SPARK-2425 |
Standalone Master is too aggressive in removing Applications |
Critical |
Spark Core |
Mark Hamstra |
Mark Hamstra |
| SPARK-2334 |
Attribute Error calling PipelinedRDD.id() in pyspark |
Major |
PySpark |
Diana Carroll |
|
| SPARK-2314 |
RDD actions are only overridden in Scala, not java or python |
Major |
SQL |
Michael Armbrust |
Aaron Staple |
| SPARK-2269 |
Clean up and add unit tests for resourceOffers in MesosSchedulerBackend |
Major |
Mesos |
Patrick Wendell |
Tim Chen |
| SPARK-2220 |
Fix remaining Hive Commands |
Major |
SQL |
Michael Armbrust |
Cheng Lian |
| SPARK-2219 |
AddJar doesn't work |
Major |
SQL |
Michael Armbrust |
Cheng Lian |
| SPARK-2192 |
Examples Data Not in Binary Distribution |
Major |
Build, MLlib |
Pat McDonough |
Sean Owen |
| SPARK-2189 |
Method for removing temp tables created by registerAsTable |
Critical |
SQL |
Michael Armbrust |
Cheng Lian |
| SPARK-2140 |
yarn stable client doesn't properly handle MEMORY_OVERHEAD for AM |
Major |
YARN |
Thomas Graves |
Chris Cope |
| SPARK-2096 |
Correctly parse dot notations for accessing an array of structs |
Minor |
SQL |
Yin Huai |
|
| SPARK-2069 |
MIMA false positives (umbrella) |
Critical |
Build |
Patrick Wendell |
Prashant Sharma |
| SPARK-2066 |
Better error message for non-aggregated attributes with aggregates |
Critical |
SQL |
Reynold Xin |
Cheng Lian |
| SPARK-2062 |
VertexRDD.apply does not use the mergeFunc |
Major |
GraphX |
Ankur Dave |
Larry Xiao |
| SPARK-1919 |
In Windows, Spark shell cannot load classes in spark.jars (--jars) |
Major |
Windows |
Andrew Or |
Andrew Or |
| SPARK-1860 |
Standalone Worker cleanup should not clean up running executors |
Blocker |
Deploy |
Aaron Davidson |
|
| SPARK-1720 |
use LD_LIBRARY_PATH instead of -Djava.library.path |
Critical |
Deploy |
Thomas Graves |
Guoqiang Li |
| SPARK-1656 |
Potential resource leak in HttpBroadcast, SparkSubmitArguments, FileSystemPersistenceEngine and DiskStore |
Major |
Spark Core |
Shixiong Zhu |
Shixiong Zhu |
| SPARK-1652 |
Fixes and improvements for spark-submit/configs |
Blocker |
Spark Core, YARN |
Patrick Wendell |
Patrick Wendell |
| SPARK-1344 |
Scala API docs for top methods |
Major |
Documentation |
Diana Carroll |
Sean Owen |
| SPARK-1307 |
don't use term 'standalone' to refer to a Spark Application |
Major |
Documentation |
Diana Carroll |
Sean Owen |
| SPARK-1209 |
SparkHadoop{MapRed,MapReduce}Util should not use package org.apache.hadoop |
Major |
Spark Core |
Sandy Ryza |
Sean Owen |
| SPARK-1047 |
Ability to disable the spark ui server (unit tests) |
Minor |
Web UI |
Heiko Braun |
Andrew Or |
| SPARK-927 |
PySpark sample() doesn't work if numpy is installed on master but not on workers |
Minor |
PySpark |
Josh Rosen |
Matthew Farrellee |


GIT is most widely used modern version control system in the world today.
Signing up for a new GitHub account
• Visit Sign in to Github
create an account with username and password
• Follow the prompts to create your personal account or organization.
Creating a brand new repository
• Once you log in click new repository
• In the upper-right corner of any page, use the drop-down menu, and select New repository.
• For the repo name, type a short title ( ex: IS218)
• add a description of your repository. For example, "My first repository on GitHub." ( Optional)
• Choose a repository visibility.
• Select Initialize this repository with a README.
• Click create repository


Uses of Package
org.apache.qpid.protonj2.codec
-
Classes in org.apache.qpid.protonj2.codec used by org.apache.qpid.protonj2.client.impl Class Description Decoder Decode AMQP types from a byte streamDecoderState Retains state of decode either between calls or across decode iterationsEncoder Encode AMQP types into binary streamsEncoderState Retains Encoder state information either between calls or across encode iterations. -
Classes in org.apache.qpid.protonj2.codec used by org.apache.qpid.protonj2.codec Class Description DecodeException Exception thrown when a type decoder fails due to some unrecoverable error or violation of AMQP type specifications for an incoming encoded byte sequence.Decoder Decode AMQP types from a byte streamDecoderState Retains state of decode either between calls or across decode iterationsDescribedTypeDecoder Interface for all DescribedType decoder implementationsDescribedTypeEncoder Interface for all DescribedType encoder implementationsEncodeException Exception thrown when a type encoder fails due to some unrecoverable error or violation of AMQP type specifications.Encoder Encode AMQP types into binary streamsEncoderState Retains Encoder state information either between calls or across encode iterations.StreamDecoder Decode AMQP types from aInputStream
StreamDecoderState Retains state of theInputStream
based decode either between calls or across decode iterationsStreamDescribedTypeDecoder Interface for all DescribedType decoder implementationsStreamTypeDecoder Interface for an decoder of a specific AMQP Type.TypeDecoder Interface for an decoder of a specific AMQP Type.TypeEncoder Interface for an encoder of a specific AMQP Type. -
Classes in org.apache.qpid.protonj2.codec used by org.apache.qpid.protonj2.codec.decoders Class Description DecodeException Exception thrown when a type decoder fails due to some unrecoverable error or violation of AMQP type specifications for an incoming encoded byte sequence.Decoder Decode AMQP types from a byte streamDecoderState Retains state of decode either between calls or across decode iterationsDescribedTypeDecoder Interface for all DescribedType decoder implementationsEncodeException Exception thrown when a type encoder fails due to some unrecoverable error or violation of AMQP type specifications.StreamDecoder Decode AMQP types from aInputStream
StreamDecoderState Retains state of theInputStream
based decode either between calls or across decode iterationsStreamDescribedTypeDecoder Interface for all DescribedType decoder implementationsStreamTypeDecoder Interface for an decoder of a specific AMQP Type.TypeDecoder Interface for an decoder of a specific AMQP Type. -
Classes in org.apache.qpid.protonj2.codec used by org.apache.qpid.protonj2.codec.decoders.messaging Class Description DecodeException Exception thrown when a type decoder fails due to some unrecoverable error or violation of AMQP type specifications for an incoming encoded byte sequence.DecoderState Retains state of decode either between calls or across decode iterationsDescribedTypeDecoder Interface for all DescribedType decoder implementationsStreamDecoderState Retains state of theInputStream
based decode either between calls or across decode iterationsStreamDescribedTypeDecoder Interface for all DescribedType decoder implementationsStreamTypeDecoder Interface for an decoder of a specific AMQP Type.TypeDecoder Interface for an decoder of a specific AMQP Type. -
Classes in org.apache.qpid.protonj2.codec used by org.apache.qpid.protonj2.codec.decoders.primitives Class Description DecodeException Exception thrown when a type decoder fails due to some unrecoverable error or violation of AMQP type specifications for an incoming encoded byte sequence.DecoderState Retains state of decode either between calls or across decode iterationsStreamDecoderState Retains state of theInputStream
based decode either between calls or across decode iterationsStreamTypeDecoder Interface for an decoder of a specific AMQP Type.TypeDecoder Interface for an decoder of a specific AMQP Type. -
Classes in org.apache.qpid.protonj2.codec used by org.apache.qpid.protonj2.codec.decoders.security Class Description DecodeException Exception thrown when a type decoder fails due to some unrecoverable error or violation of AMQP type specifications for an incoming encoded byte sequence.DecoderState Retains state of decode either between calls or across decode iterationsDescribedTypeDecoder Interface for all DescribedType decoder implementationsStreamDecoderState Retains state of theInputStream
based decode either between calls or across decode iterationsStreamDescribedTypeDecoder Interface for all DescribedType decoder implementationsStreamTypeDecoder Interface for an decoder of a specific AMQP Type.TypeDecoder Interface for an decoder of a specific AMQP Type. -
Classes in org.apache.qpid.protonj2.codec used by org.apache.qpid.protonj2.codec.decoders.transactions Class Description DecodeException Exception thrown when a type decoder fails due to some unrecoverable error or violation of AMQP type specifications for an incoming encoded byte sequence.DecoderState Retains state of decode either between calls or across decode iterationsDescribedTypeDecoder Interface for all DescribedType decoder implementationsStreamDecoderState Retains state of theInputStream
based decode either between calls or across decode iterationsStreamDescribedTypeDecoder Interface for all DescribedType decoder implementationsStreamTypeDecoder Interface for an decoder of a specific AMQP Type.TypeDecoder Interface for an decoder of a specific AMQP Type. -
Classes in org.apache.qpid.protonj2.codec used by org.apache.qpid.protonj2.codec.decoders.transport Class Description DecodeException Exception thrown when a type decoder fails due to some unrecoverable error or violation of AMQP type specifications for an incoming encoded byte sequence.DecoderState Retains state of decode either between calls or across decode iterationsDescribedTypeDecoder Interface for all DescribedType decoder implementationsStreamDecoderState Retains state of theInputStream
based decode either between calls or across decode iterationsStreamDescribedTypeDecoder Interface for all DescribedType decoder implementationsStreamTypeDecoder Interface for an decoder of a specific AMQP Type.TypeDecoder Interface for an decoder of a specific AMQP Type. -
Classes in org.apache.qpid.protonj2.codec used by org.apache.qpid.protonj2.codec.encoders Class Description DescribedTypeEncoder Interface for all DescribedType encoder implementationsEncodeException Exception thrown when a type encoder fails due to some unrecoverable error or violation of AMQP type specifications.Encoder Encode AMQP types into binary streamsEncoderState Retains Encoder state information either between calls or across encode iterations.TypeEncoder Interface for an encoder of a specific AMQP Type. -
Classes in org.apache.qpid.protonj2.codec used by org.apache.qpid.protonj2.codec.encoders.messaging Class Description DescribedTypeEncoder Interface for all DescribedType encoder implementationsEncoderState Retains Encoder state information either between calls or across encode iterations.TypeEncoder Interface for an encoder of a specific AMQP Type. -
Classes in org.apache.qpid.protonj2.codec used by org.apache.qpid.protonj2.codec.encoders.primitives Class Description EncoderState Retains Encoder state information either between calls or across encode iterations.TypeEncoder Interface for an encoder of a specific AMQP Type. -
Classes in org.apache.qpid.protonj2.codec used by org.apache.qpid.protonj2.codec.encoders.security Class Description DescribedTypeEncoder Interface for all DescribedType encoder implementationsEncoderState Retains Encoder state information either between calls or across encode iterations.TypeEncoder Interface for an encoder of a specific AMQP Type. -
Classes in org.apache.qpid.protonj2.codec used by org.apache.qpid.protonj2.codec.encoders.transactions Class Description DescribedTypeEncoder Interface for all DescribedType encoder implementationsEncoderState Retains Encoder state information either between calls or across encode iterations.TypeEncoder Interface for an encoder of a specific AMQP Type. -
Classes in org.apache.qpid.protonj2.codec used by org.apache.qpid.protonj2.codec.encoders.transport Class Description DescribedTypeEncoder Interface for all DescribedType encoder implementationsEncoderState Retains Encoder state information either between calls or across encode iterations.TypeEncoder Interface for an encoder of a specific AMQP Type.


Train Scheduler
See When You Need to Take Your Train!
Current Train Schedule
Train Name
Destination
Frequency (min)
Next Arrival
Minutes Away
Delete
Add Train
Train Name
Destination
First Train Time (HH:mm - Military Time)
Frequency (minutes)
Submit


+++ title = "Thomas the Pole" slug = "thomas-the-pole" date = 2007-03-03T16:00:00+10:00 tags = ["fellow-travellers", "eccentric-encounters"] locations = ["mexico/playa-del-carmen"] draft = false summary = """
I saw Thomas a few days earlier in Mérida, and now it seems that he's followed me (or vice versa?) to Playa. Thomas is a Polish guy, who lives in Germany, and who's on holiday from his physics degree. He's travelled to some interesting places in his time, including Eastern Europe, which he says is so corrupt it's not funny.
""" +++I saw Thomas a few days earlier in Mérida, and now it seems that he's followed me (or vice versa?) to Playa. Thomas is a Polish guy, who lives in Germany, and who's on holiday from his physics degree. He's travelled to some interesting places in his time, including Eastern Europe, which he says is so corrupt it's not funny.


Home
OLUBOLADE DEBORAH
Projects
About Deborah
About Me
Objectives
My passion is to create solutions for businesses,
individuals and society through web development
Full Name
Olubolade Deborah Bunmi
Age
In my twenties
Skills
Web development and Editing
Proficiency
Html, Css, Javascript and PHP
.


[SYMBOL]
[A]
[B]
[C]
[D]
[E]
[F]
[G]
[H]
[I]
[J]
[K]
[L]
[M]
[N]
[O]
[P]
[Q]
[R]
[S]
[T]
[U]
[V]
[W]
[X]
[Y]
[Z]
handle_accept method
handle_close method
handle_connect method
handle_error(type, value, traceback) method
handle_expt method
handle_read method
handle_write method
Hangul syllable properties, in Python 2.0
HEAD request handler
<header> tag
help methods
helper class for MimeWriter module
hexadecimal strings, converting to/from binary data
HLS color value, using functions for converting to
host platforms, checking
HSV color values, using functions for converting to
HTML (Hypertext Markup Language)
htmllib module
2nd
htmllib parser
HTTP (Hypertext Transfer Protocol)
HTTP servers BaseHTTPServer module and
CGIHTTPServer module and
posting data to
httplib module
|


{# button for each funtions in the python script #}
{# imported javascript data for the poltly app and the colorwheel #}
{# create a div for the color wheel#}
Drag the circle inside the color wheels.Drag the circle inside the opacity bar to change how vibrant the color is.


Import and register applications in bulk. Simply provide a URI that points to the location of the properties file where the keys are formatted as type.name and the values are the URIs of the apps. For convenience, a list of out-of-the-box Stream and Task app starters is provided below, as well.


This page allows you to choose quick builds by faction, ship type, or extension, then lays them out for easy printing. It uses the actual card images from FFG. Having pre-printed quick builds reduces the setup time when trying to play a quick game of X-Wing.
Choose a faction, ship type, or extension then press the update button. Save or print using the browser's print command, choosing page size and layout matching the Print Options.
The approach for page layout is to generate a single pdf where each page is the same size and multiple builds are put on the same page if a simple cut will work. This is useful when using a printing service (e.g., Staples, FedEx) that supports simple cuts like "cut in half vertically, horizontally, or both". Examine the pdf to determine what page ranges should use what type of cuts. 11"x17" has a more efficient layout.
TL - Threat Level
GitHub project: https://github.com/tsmigiel/xwing-quickbuild


[ht::head {Building RAMADDA and NLAS}] <%nochildlist%> It is easy to build and run RAMADDA and the NLAS plugin and tools. This assumes you have SVN, ANT and a Java compiler installed on your machine.
svn co https://nlas.svn.sourceforge.net/svnroot/nlas cd nlas/trunk ant


Pāsādike bahū disvā,
bhāvitatte susaṁvute;
Isi paṇḍarasagotto,
apucchi phussasavhayaṁ.
Seeing many who inspire confidence,
personally developed and well-restrained,
the hermit Paṇḍarasagotta
asked the one known as Phussa:
“Kiṁchandā kimadhippāyā,
kimākappā bhavissare;
Anāgatamhi kālamhi,
taṁ me akkhāhi pucchito”.
“In future times,
what desire and motivation
and behavior will people have?
Please answer my question.”
“Suṇohi vacanaṁ mayhaṁ,
isipaṇḍarasavhaya;
Sakkaccaṁ upadhārehi,
ācikkhissāmyanāgataṁ.
“Listen to my words,
Paṇḍarasa the hermit,
and remember them carefully,
I will describe the future.
Kodhanā upanāhī ca,
makkhī thambhī saṭhā bahū;
Ussukī nānāvādā ca,
bhavissanti anāgate.
In the future many will be
angry and hostile,
offensive, stubborn, and devious,
jealous, holding divergent views.
Aññātamānino dhamme,
gambhīre tīragocarā;
Lahukā agarū dhamme,
aññamaññamagāravā.
Imagining they understand the depths of the teaching,
they remain on the near shore.
Superficial and disrespectful towards the teaching,
they lack respect for one another.
Bahū ādīnavā loke,
uppajjissantyanāgate;
Sudesitaṁ imaṁ dhammaṁ,
kilesessanti dummatī.
In the future
many dangers will arise in the world.
Idiots will defile
the Dhamma that was taught so well.
Guṇahīnāpi saṅghamhi,
voharantā visāradā;
Balavanto bhavissanti,
mukharā assutāvino.
Though bereft of good qualities,
unlearned prattlers, too sure of themselves,
will become powerful
in running Saṅgha proceedings.
Guṇavantopi saṅghamhi,
voharantā yathātthato;
Dubbalā te bhavissanti,
hirīmanā anatthikā.
Though possessing good qualities,
the conscientious and unbiased, acting in the proper spirit,
will become weak
in running Saṅgha proceedings.
Rajataṁ jātarūpañca,
khettaṁ vatthumajeḷakaṁ;
Dāsidāsañca dummedhā,
sādiyissantyanāgate.
In the future, fools will accept
money, gold, and silver,
fields and land, goats and sheep,
and bonded servants, male and female.
Ujjhānasaññino bālā,
sīlesu asamāhitā;
Unnaḷā vicarissanti,
kalahābhiratā magā.
Fools looking for fault in others,
but unsteady in their own ethics,
will wander about, insolent,
like cantankerous beasts.
Uddhatā ca bhavissanti,
nīlacīvarapārutā;
Kuhā thaddhā lapā siṅgī,
carissantyariyā viya.
They’ll be arrogant,
wrapped in robes of blue;
deceivers, stubborn, flatterers, and frauds,
they’ll wander as if they were noble ones.
Telasaṇṭhehi kesehi,
capalā añjanakkhikā;
Rathiyāya gamissanti,
dantavaṇṇikapārutā.
With hair sleeked back with oil,
fickle, their eyes painted with eye-liner,
they’ll travel on the high-road,
wrapped in robes of ivory color.
Ajegucchaṁ vimuttehi,
surattaṁ arahaddhajaṁ;
Jigucchissanti kāsāvaṁ,
odātesu samucchitā.
The deep-dyed ocher robe,
worn without disgust by the free,
they will come to loathe,
besotted by white clothes.
Lābhakāmā bhavissanti,
kusītā hīnavīriyā;
Kicchantā vanapatthāni,
gāmantesu vasissare.
They’ll want lots of possessions,
and be lazy, lacking energy.
Weary of the forest,
they’ll stay in the neighborhood of villages.
Ye ye lābhaṁ labhissanti,
micchājīvaratā sadā;
Te teva anusikkhantā,
bhajissanti asaṁyatā.
Being unrestrained, they’ll keep company with
those who get lots of stuff,
and who always enjoy wrong livelihood,
following their example.
Ye ye alābhino lābhaṁ,
na te pujjā bhavissare;
Supesalepi te dhīre,
sevissanti na te tadā.
They won’t respect those
who don’t get lots of stuff,
and they won’t associate with the wise,
even though they’re very amiable.
Milakkhurajanaṁ rattaṁ,
garahantā sakaṁ dhajaṁ;
Titthiyānaṁ dhajaṁ keci,
dhārissantyavadātakaṁ.
Disparaging their own banner,
which is dyed the color of copper,
some will wear the white banner
of those who follow other paths.
Agāravo ca kāsāve,
tadā tesaṁ bhavissati;
Paṭisaṅkhā ca kāsāve,
bhikkhūnaṁ na bhavissati.
Then they’ll have no respect
for the ocher robe.
The mendicants will not reflect
on the nature of the ocher robe.
Abhibhūtassa dukkhena,
sallaviddhassa ruppato;
Paṭisaṅkhā mahāghorā,
nāgassāsi acintiyā.
This awful lack of reflection
was unthinkable to the elephant,
who was overcome by suffering,
injured, pierced by an arrow.
Chaddanto hi tadā disvā,
surattaṁ arahaddhajaṁ;
Tāvadeva bhaṇī gāthā,
gajo atthopasaṁhitā.
Then the six-tusked elephant,
seeing the deep-dyed banner of the perfected ones,
straight away spoke these verses
connected with the goal.
‘Anikkasāvo kāsāvaṁ,
yo vatthaṁ paridhassati;
Apeto damasaccena,
na so kāsāvamarahati.
One who, not free of stains themselves,
would wear the robe stained in ocher,
bereft of taming and truth:
they are not worthy of the ocher robe.
Yo ca vantakasāvassa,
sīlesu susamāhito;
Upeto damasaccena,
sa ve kāsāvamarahati.
One who’s purged all their stains,
steady in ethics,
possessing truth and taming:
they are truly worthy of the ocher robe.
Vipannasīlo dummedho,
pākaṭo kāmakāriyo;
Vibbhantacitto nissukko,
na so kāsāvamarahati.
Devoid of virtue, unintelligent,
wild, doing what they like,
their minds astray, indolent:
they are not worthy of the ocher robe.
Yo ca sīlena sampanno,
vītarāgo samāhito;
Odātamanasaṅkappo,
sa ve kāsāvamarahati.
One accomplished in ethics,
free of greed, serene,
their heart’s intention pure:
they are truly worthy of the ocher robe.
Uddhato unnaḷo bālo,
sīlaṁ yassa na vijjati;
Odātakaṁ arahati,
kāsāvaṁ kiṁ karissati.

Bhikkhū ca bhikkhuniyo ca,
duṭṭhacittā anādarā;
Tādīnaṁ mettacittānaṁ,
niggaṇhissantyanāgate.
In the future, monks and nuns
with corrupt hearts, lacking regard for others,
will disparage those
with hearts of loving-kindness.
Sikkhāpentāpi therehi,
bālā cīvaradhāraṇaṁ;
Na suṇissanti dummedhā,
pākaṭā kāmakāriyā.
Though trained in wearing the robe
by senior monks,
the unintelligent will not listen,
wild, doing what they like.
Te tathā sikkhitā bālā,
aññamaññaṁ agāravā;
Nādiyissantupajjhāye,
khaḷuṅko viya sārathiṁ.
With that kind of attitude to training,
those fools won’t respect each other,
or take any notice of their mentors,
like a wild colt with its charioteer.
Evaṁ anāgataddhānaṁ,
paṭipatti bhavissati;
Bhikkhūnaṁ bhikkhunīnañca,
patte kālamhi pacchime.’
Even so, in the future,
this will be the practice
of monks and nuns
when the latter days have come.
Purā āgacchate etaṁ,
anāgataṁ mahabbhayaṁ;
Subbacā hotha sakhilā,
aññamaññaṁ sagāravā.
Before this frightening future arrives,
be easy to admonish,
kind in speech,
and respect one another.
Mettacittā kāruṇikā,
Hotha sīlesu saṁvutā;
Āraddhavīriyā pahitattā,
Niccaṁ daḷhaparakkamā.
Have hearts of love and compassion,
and please do keep your precepts.
Be energetic, resolute,
and always staunchly vigorous.
Pamādaṁ bhayato disvā,
Appamādañca khemato;
Bhāvethaṭṭhaṅgikaṁ maggaṁ,
Phusantā amataṁ padan”ti.
… Phusso thero ….
Seeing negligence as fearful,
and diligence as a sanctuary,
develop the eightfold path,
realizing the deathless state.”
…


Preparation time
less than 30 mins
Cooking time
10 to 30 mins
Serves
Makes 16 rolls
I make these often – sometimes the sausage shoots rather rudely out of its skin, sometimes it doesn’t. To guarantee your sausage does not escape its roll during cooking, the sausage ‘skin’ can be slipped off before encasing it in pastry.
Preheat the oven to 200C/400F/Gas 6.
Roll the pastry out on a floured surface to a rectangle of about 48x32cm/19x12½in and bash the pastry with the rolling pin a bit. Puff pastry is made of fine layers and normally you have to be very delicate with it. For sausage rolls the pastry needs to be slightly puffed, but not too much, so bashing it with a rolling pin reduces the amount it puffs up.
Cut the large rectangle in half lengthways, then cut both smaller rectangles into eight equal sections. You now have 16 rectangles in total. Brush one end of each rectangle with a little of the beaten egg, lay a piece of sausage at the other end, then season the sausage with salt and freshly ground black pepper and sprinkle with thyme leaves. Roll the sausage up in the pastry to enclose and repeat with all the sausages. Put the sausage rolls in the fridge for 20 minutes for the pastry to harden.
Once the pastry is hard, remove the sausage rolls from the fridge and score the tops with a sharp knife for decoration, or prick with a fork. Brush well all over with the rest of the beaten egg and bake in the oven for 25–30 minutes, or until the pastry has turned golden-brown and looks crisp. Remove from the oven and leave to cool slightly before serving.


Gateway
BOT
22-Apr-19 12:30 PM
[dscrdvrsc] <Thoskk> I think mine would still work, I just have an issue with the floppy, it's on endless loop ... guess one of the ICs should be replaced.
[dscrdvrsc] <Oink> and cleaned!
[dscrdvrsc] <Oink> clickety clickety click...
(edited)
[dscrdvrsc] <Thoskk> already did that, didn't help
[dscrdvrsc] <Thoskk> and I'm not a master in soldering, so I refrained from removing the IC myself
(edited)
[dscrdvrsc] <Oink> oh, that was another device... That was the ZIP drive... Click of death...
(edited)
[dscrdvrsc] <Thoskk> oh, ZIPs ... I hated them. Totally unstable and always broken when you needed them.
[dscrdvrsc] <Oink> I har a firewire version, worked reliably for 5 or 6 years...
(edited)
[dscrdvrsc] <Thoskk> we just had SCSIs
[dscrdvrsc] <Thoskk> external
[dscrdvrsc] <Oink> My father in law had the combined SCSI/parallel version
[dscrdvrsc] <Oink> That was a disaster!
[dscrdvrsc] <Thoskk> probably same ... just remember SCSI, but may have been that one though
[dscrdvrsc] <Oink> You had a parallel, SCSI and a combined one... That was about ??? al lot more expensive
[dscrdvrsc] <Thoskk> we had them ad university, for data backup ... well, was more of data breakdown after all
[dscrdvrsc] <Oink> The firewire I had from work: one of my colleagues was a soldering whizzkid and replaced the parallel interface for a firewire interface for me
[dscrdvrsc] <Thoskk> we switched to tape instead
[dscrdvrsc] <Oink> After that you had the Jazz-drive... I never thoughed that one, though
[dscrdvrsc] <Thoskk> those had enormous capacities, compared to other media at that time
(edited)
[dscrdvrsc] <Oink> Tapes were just as unreliable.... Our Sa's never bothered to verify the backup every once in a while....
[dscrdvrsc] <Thoskk> did he? Hm, never had issues with those
(edited)
[dscrdvrsc] <Oink> And when they needed them.... They turned out to be corrupt...
[dscrdvrsc] <Thoskk> the trick was an antimagnetic casing ^^
[dscrdvrsc] <Thoskk> or rather a magnetic shielding, to put that in proper terms
[dscrdvrsc] <Oink> If you don't regularly replace the tapes with fresh tapes... After 4 years of continuoes use you cannot expect them to hold all the data
[dscrdvrsc] <Thoskk> hm, can't remember a single tape that was corrupt... we used them for about 5 years before CDs came up
[dscrdvrsc] <Thoskk> but those were much less reliable
[dscrdvrsc] <Thoskk> especially the first versions
[dscrdvrsc] <Oink> We reccommended to replace your DLT tapes every 2 years for a fresh set
[dscrdvrsc] <Oink> We never really used CD for backups, unless a customer wanted a copy. Then we'ld supply them a CD/DVD
[dscrdvrsc] <Thoskk> one of the Professors recommended to print the data, which he considered to be the safest way... we made a joke and made a printout of a program in assembler code and asked him if he could put it back into the computer ... after that, the ZIP drives came in ^^
[dscrdvrsc] <Oink> Hihihihi... Just supply the good professor with a thermal printer... Gnagnagna
[dscrdvrsc] <Thoskk> hot needles... oh yeah
[dscrdvrsc] <Oink> Just like your cash receipt from a supermarket... Leave it in the car for a week during summer.... Not readable!
[dscrdvrsc] <Thoskk> doesn't need summer, a strong source of light will be enough
[dscrdvrsc] <Oink> indeed.... I would love to see that professors face when he finds out!
[dscrdvrsc] <Thoskk> or a weak source and time
[dscrdvrsc] <Thoskk> yeah, that's also what I'm always angry about ... you need the bill for guarantee things, but after a year it's bleached ... great
[dscrdvrsc] <Oink> I always scan an copy bills I may need for warranty in the future
[dscrdvrsc] <Oink> And staple the original to the copy
(edited)
[dscrdvrsc] <Thoskk> i just put them in aluminum foil... also works ... but it's not a treat anyway
(edited)
[dscrdvrsc] <Oink> But, proof of payment should be sufficient, so if you can produce a bankstatement with the payment on it, it should suffice.... Most stores don't accept it without a fuss, though
(edited)
[dscrdvrsc] <Thoskk> luckily most cases happen within the first 2 or 3 weeks, and at least the discounters don't make much fuzz if you bring their stuff back... they just pay you back and put it back into the shelf again...
[dscrdvrsc] <Oink> Yeah, indeed. At those places I rummage through until I find a box that isn't tampered with
[dscrdvrsc] <Thoskk> ok, time for coffee and sun... l8er m8 ^^


CTFProxy
{{ .Title }}
{{ .Description }}
Debugging
Copy text
Copy this text for help with troubleshooting.
Don't use a screenshot.
error code: {{ .Code }}
type: {{ .Type }}
{{ .Debug }}


| 011:061 Khan | : |
And to Thamud (people, We sent) their brother Salih (Saleh). He said: "O my people! Worship Allah, you have no other Ilah (God) but Him. He brought you forth from the earth and settled you therein, then ask forgiveness of Him and turn to Him in repentance. Certainly, my Lord is Near (to all by His Knowledge), Responsive." |
| 011:061 Maulana | : |
And to Thamud (We sent) their brother Salih. He said: O my people, serve Allah, you have no god other than Him. He brought you forth from the earth and made you dwell in it, so ask forgiveness of Him, then turn to Him. Surely my Lord is Nigh, Answering. |
| 011:061 Pickthal | : |
And unto (the tribe of) Thamud (We sent) their brother Salih. He said: O my people!  He brought you forth from the earth and hath made you husband it. So ask forgiveness of Him and turn unto Him repentant. Lo! my Lord is Nigh, Responsive. |
| 011:061 Rashad | : |
To Thamoud we sent their brother Saaleh. He said, "O my people, worship GOD; you have no other god beside Him. He initiated you from the earth, then settled you in it. You shall seek His forgiveness, then repent to Him. My Lord is always near, responsive." |
| 011:061 Sarwar | : |
To Thamud We sent their brother Salih who told them, "My people, worship God; He is your only Lord. It is He who has created you from the earth and has settled you therein. Seek forgiveness from Him and turn to Him in repentance. My Lord is certainly close to everyone and He hears all prayers." |
| 011:061 Shakir | : |
And to Samood (We sent) their brother Salih. He said: O my people! serve Allah, you have no god other than He; He brought you into being from the earth, and made you dwell in it, therefore ask forgiveness of Him, then turn to Him; surely my Lord is Nigh, Answering. |
| 011:061 Sherali | : |
And to the tribe of Thamud, WE sent their brother Salih. He said, 'O my people, worship ALLAH; you have no god other than HIM. HE raised you up from the earth and settled you therein. So ask forgiveness of HIM, then turn to HIM wholly. Verily, my Lord is nigh, and answers prayers.' |
| 011:061 Yusufali | : |
To the Thamud People (We sent) Salih, one of their own brethren. He said: "O my people! Worship Allah: ye have no other god but Him. It is He Who hath produced you from the earth and settled you therein: then ask forgiveness of Him, and turn to Him (in repentance): for my Lord is (always) near, ready to answer." |
Concepts/Themes Covered: أركان الإسلام > التوحيد > توحيد الله تعالي > ربوبيته جلّ وعلا Pillars of Islam> Islamic> Oneness of Allah> His Divinity, the Great and Almighty أركان الإسلام > التوحيد > توحيد الله تعالي > صفات الله تعالى > القريب Pillars of Islam> Islamic> Oneness of Allah> His Glory's Characteristics> The Near أركان الإسلام > التوحيد > توحيد الله تعالي > صفات الله تعالى > المجيب Pillars of Islam> Islamic> Oneness of Allah> His Glory's Characteristics> Responsive القصص والتاريخ > (ثمود (قوم صالح The Stories and the History>Thamoud(Salih People)
This tool uses eight English parallel translations for the Holy Quran imported from Mr. Clay Smith. You can also visit Qurany Concepts for browsing the Quran themes or topics. If you want to search the Holy Quran for keywords, please visit Qurany Keywords.


Preparation time
less than 30 mins
Cooking time
30 mins to 1 hour
Serves
Makes 1
Dietary
A true British classic, Bakewell tart is best served slightly warm with a dollop of lightly whipped cream.
For the pastry, place the flour, salt, sugar, butter and almonds into a food processor and pulse until the mixture resembles breadcrumbs.
Add the eggs and egg yolk one at a time and pulse until a smooth dough is formed.
Wrap the dough in cling film and chill in the fridge for half an hour.
Preheat the oven to 180C/160C Fan/Gas 4.
Grease and flour a 23cm/9in loose-bottomed tart tin.
Unwrap the chilled pastry and roll out onto a cold, floured work surface. (NB: this is a very 'short' pastry and at first will be difficult to handle).
Line the tart tin with the pastry and trim off any excess. Prick the base with a fork, then cover the pastry lightly with some baking parchment. Cover the parchment with baking beans or uncooked rice and place into the oven to bake blind for 15 minutes.
Remove from the oven and remove the baking beans or rice and the baking parchment and set aside to cool. If the pastry base is still a little raw, return to the oven for a minute or two to dry out.
Reduce the oven temperature to 165C/145C Fan/Gas 2.
Meanwhile, for the filling, place the ground almonds and caster sugar into a bowl and mix well.
Add the beaten eggs and almond essence and mix well.
When the pastry has cooled spread a generous layer of raspberry jam onto the pastry base.
Pour the filling mixture over the raspberry jam to fill the pastry case.
Top with flaked almonds, then bake in the oven for 25-30 minutes, or until the filling is baked through and golden-brown all over.
Remove from the oven and allow to cool. Serve warm or cold.


Chapter 3 Generating functions
3.1 The moment generating function
We now define an entity — the moment generating function (mgf) — that enables us to find as many moments as we wish using just a single integral (or sum in the discrete case).
We define the moment generating function for \(Y\) as \[M_Y(t) = E(e^{tY}) = \int_{-\infty}^\infty e^{ty} f(y) dy,\] if \(Y\) has continuous distribution, or \[M_Y(t) = E(e^{tY}) = \sum_{y \in D} e^{ty} p(y)\] if \(Y\) has discrete distribution.
A problem with this definition is that in some cases the expectation \(E(e^{tY})\) might not be well-defined for some values of \(t\). Provided that there is some value \(h > 0\) such that the expectation \(E(e^{tY})\) exists for all \(-h < t < h\), we say the mgf is well-defined.
Consider \(e^{ty}\) expanded as a power series in \(t\): \[e^{ty} = 1 + ty + \frac{(ty)^2}{2!} + \frac{(ty)^3}{3!} + \ldots = \sum_{k=0}^\infty \frac{(ty)^k}{k!}.\] We can use this power series expansion to see that \[\begin{equation} M_Y(t) = E(e^{tY}) = E\left\{ \sum_{k=0}^\infty \frac{t^k Y^k}{k!} \right\} = \sum_{k=0}^\infty \frac{t^k E(Y^k)}{k!} = \sum_{k=0}^\infty \frac{t^k \mu_k^\prime}{k!}. \tag{3.1} \end{equation}\]The moment generating function allows us to easily find any moment \(\mu_r^\prime\), by using the following result.
by the binomial theorem.
To find \(E(Y)\), we first differentiate \(M_Y(t)\), to find \[M_Y^{(1)}(t) = \frac{d}{dt} M_Y(t) = \frac{d}{dt} (\theta e^t + 1 - \theta)^n = n (\theta e^t + 1 - \theta)^{n-1} \theta e^t.\] We get \[E(Y) = M_Y^{(1)}(0) = n(\theta + 1 - \theta)^{n-1} \theta = n \theta,\] as expected.
To find \(\text{Var}(Y)\), we first find \(E(Y^2)\) by differentiating the mgf a section time. We have \[\begin{align*} M_Y^{(2)}(t) &= \frac{d}{dt} n (\theta e^t + 1 - \theta)^{n-1} \theta e^t \\ & = n (n - 1) (\theta e^t + 1 - \theta)^{n-2} (\theta e^t)^2 + n (\theta e^t + 1 - \theta)^{n-1} \theta e^t. \end{align*}\] We get \[E(Y^2) = M_Y^{(2)}(0) = n (n - 1) \theta^2 + n \theta,\] so \[\begin{align*} \text{Var}(Y) &= E(Y^2) - [E(Y)]^2 \\ &= n (n - 1) \theta^2 + n \theta - (n \theta)^2 \\ &= -n \theta^2 + n \theta = n \theta(1 - \theta), \end{align*}\] as expected.where we have used the fact that the integral of a \(N(\mu + \sigma^2 t, \sigma^2)\) pdf over the whole real line is one.
To find \(E(Y)\), we first differentiate \(M_Y(t)\), to find \[\begin{align*} M_Y^{(1)}(t) &= \frac{d}{dt} \exp \left\{\mu t + \frac{\sigma^2 t^2}{2} \right\} \\ &= (\mu + \sigma^2 t) \exp \left\{\mu t + \frac{\sigma^2 t^2}{2} \right\}. \end{align*}\]So \(E(Y) = M_Y^{(1)}(0) = \mu\), as expected.
To find \(\text{Var}(Y)\), we first find \(E(Y^2)\) by differentiating the mgf a section time. We have \[\begin{align*} M_Y^{(2)}(t) &= \frac{d}{dt} (\mu + \sigma^2 t) \exp \left\{\mu t + \frac{\sigma^2 t^2}{2} \right\} \\ &= \sigma^2 \exp \left\{\mu t + \frac{\sigma^2 t^2}{2} \right\} + (\mu + \sigma^2 t)^2 \exp \left\{\mu t + \frac{\sigma^2 t^2}{2} \right\} \end{align*}\] We get \(E(Y^2) = M_Y^{(2)}(0) = \sigma^2 + \mu^2,\) so \[\text{Var}(Y) = E(Y^2) - [E(Y)]^2 = \sigma^2 + \mu^2 - \mu^2 = \sigma^2,\] as expected.3.2 The cumulant generating function
Define the cumulant generating function (cgf) by \[K_Y(t) = \log M_Y(t).\] Define the \(r\)th cumulant \(\kappa_r\) by \[\kappa_r = K_Y^{(r)}(0).\] What are these cumulants in terms of the more familiar moments?
We have \[K_Y^{(1)}(t) = \frac{d}{dt} \log M_Y(t) = \frac{M_Y^{(1)}(t)}{M_Y(t)},\] so \[\kappa_1 = K_Y^{(1)}(0) = \frac{M_Y^{(1)}(0)}{M_Y(0)} = \frac{\mu_1^\prime}{1},\] so \(\kappa_1 = \mu_1^\prime = \mu\).
We have \[K_2^{(1)}(t) = \frac{d}{dt}\frac{M_Y^{(1)}(t)}{M_Y(t)} = \frac{M_Y^{(2)}(t)}{M_Y(t)} - \frac{\left[M_Y^{(1)}(t)\right]^2}{\left[M_Y(t)\right]^2},\] so \[\kappa_2 = K_Y^{(2)}(0) = \frac{\mu_2^\prime}{1} - \frac{\left(\mu_1^\prime\right)^2}{1^2} = \mu_2^\prime - \mu^2 = \mu_2 = \text{Var}(Y).\] So we can find \(\text{Var}(Y)\) directly by differentiating the cumulant generating function.
In a similar manner, we can show that \(\kappa_3 = \mu_3\), so we can find the third central moment directly from the cgf.
It is tempting to assume that \(\kappa_4 = \mu_4\), but this is not the case. In fact, we can show that \[\kappa_4 = \mu_4 - 3 \mu_2^2,\] so \(\mu_4 = \kappa_4 + 3 \mu_2^2 = \kappa_4 + 3 \kappa_2^2\).
So we see that if we are just interested in finding the mean, variance, skewness and kurtosis of a distribution, then the cgf is particularly useful.
Example 3.4 (Normal cgf) Suppose that \(Y \sim N(\mu, \sigma^2)\). From Example 3.2, we know that \(M_Y(t) = \exp\{\mu t + \frac{1}{2} \sigma^2 t^2\}\), so \(K_Y(t) = \mu t + \frac{1}{2} \sigma^2 t^2\).
By differentiating this, we get \(\kappa_1 = \mu\), \(\kappa_2 = \sigma^2\), and \(\kappa_r = 0\) for \(r = 3, 4, 5, \ldots\).
The third moment about the mean is \(\mu_3 = 0\), so the skewness is \(\gamma_1 = 0\) for all normal random variables.
The fourth moment about the mean is \(\mu_4 = \kappa_4 + 3 \kappa_2^2 = 0 + 3 \sigma^4\), so the kurtosis is \(\gamma_2 = 3\) for all normal random variables.3.3 Generating functions under linear transformation
Using this result, we see that some distributions are “closed” under certain types of linear transformation.
Clearly, if two random variables have the same distribution, then they have the same mgf (cgf). We also state without proof that if two random variables have the same mgf (cgf), then they have the same distribution. We say that the mgf and cgf characterise the distribution: if we find the mgf (cgf) of a random variable, and find that it matches the mgf (cgf) of a known distribution, we can immediately conclude that the random variable has that distribution.
Example 3.5 (Scaling an exponential distribution) Suppose \(Y \sim \text{exponential}(\theta)\). Let \(Z = bY\) where \(b > 0\). Then we claim that \(Z \sim \text{exponential}(\theta / b)\).
The mgf of \(Y\) is \[\begin{align*} M_Y(t) = E(e^{tY}) &= \int_{0}^\infty e^ty \theta e^{-\theta y} \\ &= \theta \int_0^\infty e^{-(\theta - t)y} dy \\ &= \frac{\theta}{\theta - t} \text{if $\theta - t > 0$, i.e. if $t < \theta$.} \end{align*}\] By Theorem 3.2 with \(a = 0\), we have \[M_Z(t) = M_Y(bt) = \frac{\theta}{\theta - bt} = \frac{\theta/ b}{\theta/b - t},\] which is the \(\text{exponential}(\theta/b)\) mgf. Since the mgf characterises the distribution, we conclude \(Z \sim \text{exponential}(\theta / b)\). A scale change of any exponential random variable gives another exponential random variable.Example 3.6 (Linear transformation of a Normal distribution) Suppose \(Y \sim N(\mu, \sigma^2)\). Let \(Z = a + bY\) where \(a \in \mathbb{R}\) and \(b > 0\). Then we claim that \(Z \sim N(a + b \mu, b^2 \sigma^2)\).
From Example 3.4, we know that \(K_Y(t) = \mu t + \frac{1}{2} \sigma^2 t^2\). By Theorem 3.2, we have \[K_Z(t) = at + K_Y(bt) = at + \mu b t + \frac{1}{2} \sigma^2 (b t)^2 = (a + b \mu) t + \frac{1}{2}(b \sigma)^2 t^2,\] which is the \(N(a + b\mu, b^2 \sigma^2)\) cgf. Since the cgf characterises the distribution, we conclude \(Z \sim N(a + b\mu, b^2 \sigma^2)\). A linear transformation of any normal random variable gives another normal random variable.