| JIRA | Summary | Priority | Component | Reporter | Contributor |
|---|---|---|---|---|---|
| SPARK-12016 | word2vec load model can't use findSynonyms to get words | Major | PySpark | yuangang.liu | Liang-Chi Hsieh |
| SPARK-13195 | PairDStreamFunctions.mapWithState fails in case timeout is set without updating State[S] | Major | Streaming | Yuval Itzchakov | Shixiong Zhu |
| SPARK-12555 | Datasets: data is corrupted when input data is reordered | Major | SQL | Tim Preece | Luciano Resende |
| SPARK-14897 | Upgrade Jetty to latest version of 8/9 | Major | . | Adam Kramer | Bo Meng |
| SPARK-15652 | Missing org.apache.spark.launcher.SparkAppHandle.Listener notification if SparkSubmit JVM shutsdown | Critical | . | Subroto Sanyal | Subroto Sanyal |
| SPARK-15851 | Spark 2.0 does not compile in Windows 7 | Major | Build | Alexander Ulanov | Reynold Xin |
| SPARK-13709 | Spark unable to decode Avro when partitioned | Major | SQL | Chris Miller | Cheng Lian |
| JIRA | Summary | Priority | Component | Reporter | Contributor |
|---|---|---|---|---|---|
| SPARK-11206 | Support SQL UI on the history server | Major | SQL, Web UI | Carson Wang | Carson Wang |
| SPARK-6990 | Add Java linting script | Minor | Project Infra | Josh Rosen | Dmitry Erastov |
| SPARK-11774 | Implement "struct", "encode","decode" in SparkR | Major | SparkR | Sun Rui | Sun Rui |
| SPARK-11713 | Initial RDD for updateStateByKey for pyspark | Major | PySpark | David Watson | Bryan Cutler |
| SPARK-9516 | Improve Thread Dump page | Minor | Web UI | Nan Zhu | Nan Zhu |
| SPARK-12321 | JSON format for logical/physical execution plans | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-10359 | Enumerate Spark's dependencies in a file and diff against it for new pull requests | Major | Build, Project Infra | Patrick Wendell | Josh Rosen |
| SPARK-12541 | Support rollup/cube in SQL query | Major | SQL | Davies Liu | Davies Liu |
| SPARK-12542 | Support intersect/except in Hive SQL | Major | SQL | Davies Liu | Davies Liu |
| SPARK-10809 | Single-document topicDistributions method for LocalLDAModel | Minor | MLlib | Joseph K. Bradley | yuhao yang |
| SPARK-9843 | Catalyst: Allow adding custom optimizers | Major | SQL | Robert Kruszewski | Robert Kruszewski |
| SPARK-12785 | Implement columnar in memory representation | Major | SQL | Nong Li | Nong Li |
| SPARK-12538 | bucketed table support | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-12394 | Support writing out pre-hash-partitioned data and exploit that in join optimizations to avoid shuffle (i.e. bucketing in Hive) | Major | SQL | Reynold Xin | Nong Li |
| SPARK-12796 | initial prototype: projection/filter/range | Major | SQL | Davies Liu | Davies Liu |
| SPARK-2750 | Add Https support for Web UI | Major | Web UI | Tao Wang | Fei Wang |
| SPARK-12337 | Implement dropDuplicates() method of DataFrame in SparkR | Major | SparkR | Sun Rui | Sun Rui |
| SPARK-6519 | Add spark.ml API for bisecting k-means | Major | ML | Yu Ishikawa | Yu Ishikawa |
| SPARK-12797 | Aggregation without grouping keys | Major | SQL | Davies Liu | Davies Liu |
| SPARK-12204 | Implement drop method for DataFrame in SparkR | Major | SparkR | Sun Rui | Sun Rui |
| SPARK-12818 | Implement Bloom filter and count-min sketch in DataFrames | Major | SQL | Reynold Xin | Cheng Lian |
| SPARK-9835 | Iteratively reweighted least squares solver for GLMs | Critical | ML, MLlib | Xiangrui Meng | Yanbo Liang |
| SPARK-12401 | Add support for enums in postgres | Minor | SQL | Jaka Jancar | Takeshi Yamamuro |
| SPARK-12798 | Broadcast hash join | Major | SQL | Davies Liu | Davies Liu |
| SPARK-3611 | Show number of cores for each executor in application web UI | Minor | Web UI | Matei Zaharia | Alex Bozarth |
| SPARK-12828 | support natural join | Major | SQL | Adrian Wang | Adrian Wang |
| SPARK-13074 | Add getPersistentRDDs() API to JavaSparkContext | Minor | Spark Core | Junyang Shen | Junyang Shen |
| SPARK-11515 | QuantileDiscretizer should take random seed | Minor | ML | Joseph K. Bradley | Yu Ishikawa |
| SPARK-12544 | Support window functions in SQLContext | Major | SQL | Davies Liu | Herman van Hovell |
| SPARK-13075 | Native database/table system catalog | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-13373 | Generate code for sort merge join | Major | SQL | Davies Liu | Davies Liu |
| SPARK-13011 | K-means wrapper in SparkR | Major | ML, SparkR | Xiangrui Meng | Xusen Yin |
| SPARK-13479 | Python API for DataFrame approxQuantile | Minor | PySpark, SQL | Joseph K. Bradley | Joseph K. Bradley |
| SPARK-13292 | QuantileDiscretizer should take random seed in PySpark | Minor | ML, PySpark | Xiangrui Meng | Yu Ishikawa |
| SPARK-13028 | Add MaxAbsScaler to ML.feature as a transformer | Minor | ML | yuhao yang | yuhao yang |
| SPARK-13504 | Add approxQuantile for SparkR | Major | SparkR | Yanbo Liang | Yanbo Liang |
| SPARK-13465 | Add a task failure listener to TaskContext | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-13505 | Python API for MaxAbsScaler | Major | ML, PySpark | Xiangrui Meng | Li Ping Zhang |
| SPARK-12811 | Estimator interface for generalized linear models (GLMs) | Critical | ML | Xiangrui Meng | Yanbo Liang |
| SPARK-12877 | TrainValidationSplit is missing in pyspark.ml.tuning | Major | PySpark | Wojciech Jurczyk | Jeremy |
| SPARK-13543 | Support for specifying compression codec for Parquet/ORC via option() | Minor | SQL | Hyukjin Kwon | Hyukjin Kwon |
| SPARK-13523 | Reuse the exchanges in a query | Major | SQL | Davies Liu | Davies Liu |
| SPARK-11861 | Expose feature importances API for decision trees | Minor | ML | Seth Hendrickson | Seth Hendrickson |
| SPARK-13787 | Feature importances for decision trees in Python | Minor | ML, PySpark | Joseph K. Bradley | Seth Hendrickson |
| SPARK-13917 | Generate code for broadcast left semi join | Major | SQL | Davies Liu | Davies Liu |
| SPARK-9837 | Provide R-like summary statistics for GLMs via iteratively reweighted least squares | Critical | ML, MLlib | Xiangrui Meng | Yanbo Liang |
| SPARK-13927 | Add row/column iterator to local matrices | Minor | MLlib | Xiangrui Meng | Xiangrui Meng |
| SPARK-13629 | Add binary toggle Param to CountVectorizer | Minor | ML | Joseph K. Bradley | yuhao yang |
| SPARK-13764 | Parse modes in JSON data source | Minor | SQL | Hyukjin Kwon | Hyukjin Kwon |
| SPARK-13449 | Naive Bayes wrapper in SparkR | Major | ML, SparkR | Xiangrui Meng | Xusen Yin |
| SPARK-13010 | Survival analysis in SparkR | Major | ML, SparkR | Xiangrui Meng | Yanbo Liang |
| SPARK-14061 | implement CreateMap | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-14135 | Add off-heap storage memory bookkeeping support to MemoryManager | Major | Spark Core | Josh Rosen | Josh Rosen |
| SPARK-11730 | Feature Importance for GBT | Major | ML, MLlib | Brian Webb | Seth Hendrickson |
| SPARK-10570 | Add Spark version endpoint to standalone JSON API | Minor | Spark Core, Web UI | Matt Cheah | Jakob Odersky |
| SPARK-14264 | Add feature importances for GBTs in Pyspark | Minor | ML, PySpark | Seth Hendrickson | Seth Hendrickson |
| SPARK-13992 | Add support for off-heap caching | Major | Spark Core | Josh Rosen | Josh Rosen |
| SPARK-7179 | Add pattern after "show tables" to filter desire tablename | Minor | SQL | baishuo | Herman van Hovell |
| SPARK-11611 | Python API for bisecting k-means | Major | MLlib, PySpark | Xiangrui Meng | Yu Ishikawa |
| SPARK-14334 | Add toLocalIterator for Dataset | Major | SQL | Davies Liu | Davies Liu |
| SPARK-14353 | Dateset Time Windowing API for Python, R, and SQL | Major | PySpark, SparkR, SQL | Burak Yavuz | Burak Yavuz |
| SPARK-12569 | DecisionTreeRegressor: provide variance of prediction: Python API | Minor | ML, PySpark | Joseph K. Bradley | Miao Wang |
| SPARK-14392 | CountVectorizer Estimator should include binary toggle Param | Minor | ML | Joseph K. Bradley | Miao Wang |
| SPARK-14500 | Accept Dataset[_] instead of DataFrame in MLlib APIs | Major | ML | Xiangrui Meng | Xiangrui Meng |
| SPARK-12566 | GLM model family, link function support in SparkR:::glm | Critical | ML, SparkR | Joseph K. Bradley | yuhao yang |
| SPARK-13597 | Python API for GeneralizedLinearRegression | Critical | ML, PySpark | Xiangrui Meng | Kai Jiang |
| SPARK-14461 | GLM training summaries should provide solver | Minor | ML | Joseph K. Bradley | Yanbo Liang |
| SPARK-13967 | Add binary toggle Param to PySpark CountVectorizer | Minor | ML | Nick Pentreath | Bryan Cutler |
| SPARK-14545 | Improve LikeSimplification by adding a%b rule |
Major | Optimizer | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-13925 | Expose R-like summary statistics in SparkR::glm for more family and link functions | Critical | ML, SparkR | Xiangrui Meng | Yanbo Liang |
| SPARK-7861 | Python wrapper for OneVsRest | Minor | ML, PySpark | Ram Sriharsha | Xusen Yin |
| SPARK-14564 | Python Word2Vec missing setWindowSize method | Minor | ML, PySpark | Brad Willard | Jason C Lee |
| SPARK-14042 | Add support for custom coalescers | Major | Spark Core | Nezih Yigitbasi | Nezih Yigitbasi |
| SPARK-8171 | Support Javascript-based infinite scrolling in Spark log viewers | Major | Web UI | Josh Rosen | Alex Bozarth |
| SPARK-12543 | Support subquery in select/where/having | Major | SQL | Davies Liu | Herman van Hovell |
| SPARK-14776 | Merge HiveSqlAstBuilder and SparkSqlAstBuilder | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-14433 | PySpark ml GaussianMixture | Major | ML, PySpark | Joseph K. Bradley | Miao Wang |
| SPARK-13734 | SparkR histogram | Minor | SparkR | Oscar D. Lara Yejas | Oscar D. Lara Yejas |
| SPARK-11012 | Canonicalize view definitions | Major | SQL | Yin Huai | Yin Huai |
| SPARK-7264 | SparkR API for parallel functions | Major | SparkR | Shivaram Venkataraman | Timothy Hunter |
| SPARK-14412 | spark.ml ALS prefered storage level Params | Minor | ML | Joseph K. Bradley | Nick Pentreath |
| SPARK-15030 | Support formula in spark.kmeans in SparkR | Major | ML, SparkR | Xiangrui Meng | Yanbo Liang |
| SPARK-14781 | Support subquery in nested predicates | Major | SQL | Davies Liu | Davies Liu |
| SPARK-14785 | Support correlated scalar subquery | Major | . | Davies Liu | Herman van Hovell |
| SPARK-15021 | cannot run aggregate function on explode result | Major | SQL | Reynold Xin | Wenchen Fan |
| SPARK-15110 | SparkR - Implement repartitionByColumn on DataFrame | Major | SparkR | Narine Kokhlikyan | Narine Kokhlikyan |
| SPARK-11395 | Support over and window specification in SparkR | Major | SparkR | Sun Rui | Sun Rui |
| SPARK-14476 | Show table name or path in string of DataSourceScan | Critical | SQL | Davies Liu | Xiang Zhong |
| SPARK-15171 | Deprecate registerTempTable and add dataset.createTempView | Critical | . | Sean Zhong | Xiang Zhong |
| SPARK-10605 | collect_list() and collect_set() should accept struct types as argument | Major | SQL | Mike Fang | Herman van Hovell |
| SPARK-14541 | SQL function: IFNULL, NULLIF, NVL and NVL2 | Major | SQL | Davies Liu | Reynold Xin |
| SPARK-15181 | Python API for Generalized Linear Regression Summary | Major | ML, PySpark | Seth Hendrickson | Seth Hendrickson |
| SPARK-14900 | spark.ml classification metrics should include accuracy | Minor | ML | Joseph K. Bradley | Miao Wang |
| SPARK-13148 | document zero-keytab Oozie application launch; add diagnostics | Major | YARN | Steve Loughran | Steve Loughran |
| SPARK-13944 | Separate out local linear algebra as a standalone module without Spark dependency | Blocker | Build, ML | Xiangrui Meng | DB Tsai |
| SPARK-15908 | Add varargs-type dropDuplicates() function in SparkR | Major | SparkR | Sun Rui | Dongjoon Hyun |
| SPARK-16031 | Add debug-only socket source in Structured Streaming | Major | SQL, Streaming | Matei Zaharia | Matei Zaharia |
| SPARK-16074 | Expose VectorUDT/MatrixUDT in a public API | Critical | MLlib | Xiangrui Meng | Xiangrui Meng |
| SPARK-16118 | getDropLast is missing in OneHotEncoder | Major | MLlib | Xiangrui Meng | Xiangrui Meng |
| SPARK-16503 | SparkSession should provide Spark version | Major | Spark Core | Joseph K. Bradley | Liwei Lin |
| SPARK-12420 | Have a built-in CSV data source implementation | Major | SQL | Reynold Xin | |
| SPARK-9041 | Support reading of delimited text files as DataFrames | Major | SQL | Vishal Bagga |
| JIRA | Summary | Priority | Component | Reporter | Contributor |
|---|---|---|---|---|---|
| SPARK-11717 | Ignore R session and history files from git | Trivial | SparkR | Kai Sasaki | Kai Sasaki |
| SPARK-11044 | Parquet writer version fixed as version1 | Minor | SQL | Hyukjin Kwon | Hyukjin Kwon |
| SPARK-11692 | Support for Parquet logical types, JSON and BSON (embedded types) | Major | SQL | Hyukjin Kwon | Hyukjin Kwon |
| SPARK-11903 | Deprecate make-distribution.sh --skip-java-test | Minor | Build | Nicholas Chammas | Nicholas Chammas |
| SPARK-11592 | flush spark-sql command line history to history file | Minor | SQL | Adrian Wang | Adrian Wang |
| SPARK-11929 | spark-shell log level customization is lost if user provides a log4j.properties file | Minor | Spark Core, Spark Shell | Marcelo Vanzin | Marcelo Vanzin |
| SPARK-11983 | remove all unused codegen fallback traits | Minor | SQL | Adrian Wang | Adrian Wang |
| SPARK-11982 | Improve performance of CartesianProduct | Minor | SQL | Davies Liu | Davies Liu |
| SPARK-11898 | Use broadcast for the global tables in Word2Vec | Major | MLlib | yuhao yang | yuhao yang |
| SPARK-12080 | Kryo - Support multiple user registrators | Minor | Spark Core | Rotem Shaul | Rotem Shaul |
| SPARK-12096 | remove the old constraint in word2vec | Trivial | MLlib | yuhao yang | yuhao yang |
| SPARK-11988 | Update JPMML to 1.2.7 | Minor | ML, MLlib | Sean Owen | Sean Owen |
| SPARK-12115 | Change numPartitions() in RDD to be "getNumPartitions" to be consistent with pyspark/scala | Major | SparkR | Sun Rui | Yanbo Liang |
| SPARK-12044 | Fix usage of isnan, isNaN | Major | SparkR | Yanbo Liang | Yanbo Liang |
| SPARK-12032 | Filter can't be pushed down to correct Join because of bad order of Join | Critical | SQL | Davies Liu | Davies Liu |
| SPARK-12060 | Avoid memory copy in JavaSerializerInstance.serialize | Critical | Spark Core | Shixiong Zhu | Shixiong Zhu |
| SPARK-11884 | Drop multiple columns in the DataFrame API | Minor | Spark Core | Ted Yu | Ted Yu |
| SPARK-12103 | Clarify documentation of KafkaUtils createStream with multiple topics | Minor | Documentation, Streaming | Dan Dutrow | Cody Koeninger |
| SPARK-12166 | Unset hadoop related environment in testing | Minor | Tests | Jeff Zhang | Jeff Zhang |
| SPARK-11439 | Optimization of creating sparse feature without dense one | Minor | ML | Kai Sasaki | Nakul Jindal |
| SPARK-11155 | Stage summary json should include stage duration | Minor | Web UI | Imran Rashid | Xin Ren |
| SPARK-12074 | Avoid memory copy involving ByteBuffer.wrap(ByteArrayOutputStream.toByteArray) | Major | Spark Core | Ted Yu | Ted Yu |
| SPARK-10299 | word2vec should allow users to specify the window size | Minor | MLlib | holdenk | holdenk |
| SPARK-12241 | Improve failure reporting in Yarn client obtainTokenForHBase() | Minor | YARN | Steve Loughran | Steve Loughran |
| SPARK-11824 | WebUI throws console error for descriptions with 'bad' HTML | Minor | SQL, Web UI | Andy Robb | Sean Owen |
| SPARK-11530 | Return eigenvalues with PCA model | Minor | ML, MLlib | Christos Iraklis Tsatsoulis | Sean Owen |
| SPARK-12228 | Use in-memory for execution hive's derby metastore | Major | SQL | Yin Huai | Yin Huai |
| SPARK-10991 | LogisticRegressionTrainingSummary should dynamically add prediction col if there is no prediction col set | Minor | ML | holdenk | holdenk |
| SPARK-12273 | Spark Streaming Web UI does not list Receivers in order | Minor | Streaming, Web UI | Liwei Lin | Liwei Lin |
| SPARK-12213 | Query with only one distinct should not having on expand | Major | SQL | Davies Liu | Davies Liu |
| SPARK-12288 | Support UnsafeRow in Coalesce/Except/Intersect | Major | SQL | Davies Liu | Xiao Li |
| SPARK-12332 | Typo in ResetSystemProperties.scala's comments | Trivial | Tests | holdenk | holdenk |
| SPARK-12130 | Replace shuffleManagerClass with shortShuffleMgrNames in ExternalShuffleBlockResolver | Major | Shuffle, YARN | Lianhui Wang | Lianhui Wang |
| SPARK-10123 | Cannot set "--deploy-mode" in default configuration | Minor | Spark Core | Marcelo Vanzin | Saisai Shao |
| SPARK-10477 | using DSL in ColumnPruningSuite to improve readablity | Trivial | SQL, Tests | Wenchen Fan | Wenchen Fan |
| SPARK-4117 | Spark on Yarn handle AM being told command from RM | Major | YARN | Thomas Graves | Devaraj K |
| SPARK-12304 | Make Spark Streaming web UI display more friendly Receiver graphs | Minor | Streaming | Liwei Lin | Liwei Lin |
| SPARK-12249 | JDBC non-equality comparison operator not pushed down. | Major | SQL | Hyukjin Kwon | Hyukjin Kwon |
| SPARK-12314 | isnull operator not pushed down for JDBC datasource. | Major | SQL | Hyukjin Kwon | Hyukjin Kwon |
| SPARK-12315 | isnotnull operator not pushed down for JDBC datasource. | Major | SQL | Hyukjin Kwon | Hyukjin Kwon |
| SPARK-8745 | Remove GenerateProjection | Major | SQL | Reynold Xin | Davies Liu |
| SPARK-12309 | Use sqlContext from MLlibTestSparkContext for spark.ml test suites | Major | ML | Yanbo Liang | Yanbo Liang |
| SPARK-12361 | Should set PYSPARK_DRIVER_PYTHON before python test | Minor | PySpark, Tests | Jeff Zhang | Jeff Zhang |
| SPARK-12364 | Add ML example for SparkR | Major | ML, SparkR | Yanbo Liang | Yanbo Liang |
| SPARK-12164 | [SQL] Display the binary/encoded values | Major | SQL | Xiao Li | Xiao Li |
| SPARK-11904 | pyspark reduceByKeyAndWindow with invFunc=None requires checkpointing | Major | PySpark | David Tolpin | David Tolpin |
| SPARK-12057 | Prevent failure on corrupt JSON records | Minor | SQL | Ian Macalinao | Yin Huai |
| SPARK-12397 | Improve error messages for data sources when they are not found | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-12054 | Consider nullable in codegen | Major | SQL | Davies Liu | Davies Liu |
| SPARK-12411 | Reconsider executor heartbeats rpc timeout | Major | Spark Core | Nong Li | Nong Li |
| SPARK-10158 | ALS should print better errors when given Long IDs | Minor | ML, MLlib, PySpark | Joseph K. Bradley | Bryan Cutler |
| SPARK-12349 | Make spark.ml PCAModel load backwards compatible | Minor | ML | Joseph K. Bradley | Sean Owen |
| SPARK-12398 | Smart truncation of DataFrame / Dataset toString | Major | SQL | Reynold Xin | Dilip Biswal |
| SPARK-12374 | Improve performance of Range APIs via adding logical/physical operators | Critical | SQL | Xiao Li | Apache Spark |
| SPARK-12150 | numPartitions argument to sqlContext.range() should be optional | Minor | SQL | Henri DF | Xiao Li |
| SPARK-12392 | Optimize a location order of broadcast blocks by considering preferred local hosts | Major | Spark Core | Takeshi Yamamuro | Takeshi Yamamuro |
| SPARK-12388 | Change default compressor to LZ4 | Minor | SQL | Davies Liu | Davies Liu |
| SPARK-12475 | Upgrade Zinc from <IP_ADDRESS> to 0.3.9 | Major | Build, Project Infra | Josh Rosen | Josh Rosen |
| SPARK-12471 | Spark daemons should log their pid in the log file | Major | Spark Core | Nong Li | Nong Li |
| SPARK-11164 | Add InSet pushdown filter back for Parquet | Major | SQL | Liang-Chi Hsieh | Xiao Li |
| SPARK-12500 | Fix Tachyon deprecations; pull Tachyon dependency into one class | Minor | Spark Core | Sean Owen | Sean Owen |
| SPARK-12440 | Avoid setCheckpointDir warning when filesystem is not local | Trivial | Spark Core | Pierre Borckmans | Pierre Borckmans |
| SPARK-12515 | Minor clarification on DataFrameReader.jdbc doc | Trivial | Documentation, SQL | Felix Cheung | Felix Cheung |
| SPARK-12287 | Support UnsafeRow in MapPartitions/MapGroups/CoGroup | Major | SQL | Davies Liu | Xiao Li |
| SPARK-7727 | Avoid inner classes in RuleExecutor | Major | SQL | Santiago M. Mola | Stephan Kessler |
| SPARK-12490 | Don't use Javascript for web UI's paginated table navigation controls | Major | Web UI | Josh Rosen | Josh Rosen |
| SPARK-12547 | Tighten scala style checker enforcement for UDF registration | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-12549 | UDFs' input type specification should take Option[Seq[DataType]] rather than Seq[DataType] | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-12564 | Improve missing column AnalysisException | Minor | SQL | Michael Armbrust | Xiao Li |
| SPARK-12263 | IllegalStateException: Memory can't be 0 for SPARK_WORKER_MEMORY without unit | Trivial | Documentation | Jacek Laskowski | Neelesh Srinivas Salian |
| SPARK-12409 | JDBC AND operator push down | Minor | SQL | Huaxin Gao | Takeshi Yamamuro |
| SPARK-12387 | JDBC IN operator push down | Minor | SQL | Huaxin Gao | Takeshi Yamamuro |
| SPARK-12391 | JDBC OR operator push down | Minor | SQL | Huaxin Gao | Takeshi Yamamuro |
| SPARK-12585 | The numFields of UnsafeRow should not changed by pointTo() | Major | SQL | Davies Liu | Apache Spark |
| SPARK-12284 | Output UnsafeRow from window function | Major | SQL | Davies Liu | Davies Liu |
| SPARK-12294 | Support UnsafeRow in HiveTableScan | Major | SQL | Davies Liu | Davies Liu |
| SPARK-12290 | Change the default value in SparkPlan | Major | SQL | Davies Liu | Davies Liu |
| SPARK-10180 | JDBCRDD does not process EqualNullSafe filter. | Minor | SQL | Hyukjin Kwon | Hyukjin Kwon |
| SPARK-12599 | Remove the use of the deprecated callUDF in MLlib | Major | MLlib, SQL | Reynold Xin | Reynold Xin |
| SPARK-12537 | Add option to accept quoting of all character backslash quoting mechanism | Major | SQL | Cazen Lee | Cazen Lee |
| SPARK-12608 | Remove submitJobThreadPool since submitJob doesn't create a separate thread to wait for the job result | Major | Streaming | Shixiong Zhu | Shixiong Zhu |
| SPARK-11259 | Params.validateParams() should be called automatically | Minor | ML | Yanbo Liang | Yanbo Liang |
| SPARK-12289 | Support UnsafeRow in TakeOrderedAndProject/Limit | Major | SQL | Davies Liu | Davies Liu |
| SPARK-12292 | Support UnsafeRow in Generate | Major | SQL | Davies Liu | Davies Liu |
| SPARK-12293 | Support UnsafeRow in LocalTableScan | Major | SQL | Davies Liu | Davies Liu |
| SPARK-12331 | R^2 for regression through the origin | Minor | ML | Imran Younus | Imran Younus |
| SPARK-12438 | Add SQLUserDefinedType support for encoder | Major | SQL | Liang-Chi Hsieh | Liang-Chi Hsieh |
| SPARK-12643 | Set lib directory for antlr | Minor | Build | Liang-Chi Hsieh | Liang-Chi Hsieh |
| SPARK-12636 | Expose API on UnsafeRowRecordReader to just run on files | Major | SQL | Nong Li | Nong Li |
| SPARK-12450 | Un-persist broadcasted variables in KMeans | Minor | MLlib | RJ Nowling | RJ Nowling |
| SPARK-11878 | Eliminate distribute by in case group by is present with exactly the same grouping expressions | Minor | SQL | Yash Datta | Yash Datta |
| SPARK-7675 | PySpark spark.ml Params type conversions | Minor | ML, PySpark | Joseph K. Bradley | holdenk |
| SPARK-11531 | PySpark SparseVector: improve error message for bad indices | Trivial | MLlib, PySpark | Urvish Parikh | Rekha Joshi |
| SPARK-12368 | Better doc for the binary classification evaluator' metricName | Minor | Documentation, ML | Benjamin Fradet | Benjamin Fradet |
| SPARK-12663 | More informative error message in MLUtils.loadLibSVMFile | Minor | MLlib | Robert Dodier | Robert Dodier |
| SPARK-12295 | Manage the memory used by window function | Major | SQL | Davies Liu | Davies Liu |
| SPARK-12580 | Remove string concatenations from usage and extended in @ExpressionDescription | Minor | SQL | Kazuaki Ishizaki | Kazuaki Ishizaki |
| SPARK-12317 | Support configurate value for AUTO_BROADCASTJOIN_THRESHOLD and SHUFFLE_TARGET_POSTSHUFFLE_INPUT_SIZE with unit(e.g. kb/mb/gb) in SQLConf | Minor | SQL | Yadong Qi | kevin yu |
| SPARK-12510 | Refactor ActorReceiver to support Java | Major | Streaming | Shixiong Zhu | Shixiong Zhu |
| SPARK-12701 | Logging FileAppender should use join to ensure thread is finished | Minor | Spark Core | Bryan Cutler | Bryan Cutler |
| SPARK-12730 | De-duplicate some test code in BlockManagerSuite | Major | Tests | Josh Rosen | Josh Rosen |
| SPARK-12735 | Move spark-ec2 scripts to AMPLab | Major | EC2 | Reynold Xin | Reynold Xin |
| SPARK-12269 | Update aws-java-sdk version | Minor | Streaming | Brian London | Brian London |
| SPARK-12603 | PySpark MLlib GaussianMixtureModel should support single instance predict/predictSoft | Minor | MLlib, PySpark | Yanbo Liang | Yanbo Liang |
| SPARK-12498 | BooleanSimplification cleanup | Minor | SQL | Cheng Lian | Cheng Lian |
| SPARK-5273 | Improve documentation examples for LinearRegression | Minor | Documentation | Dev Lakhani | Sean Owen |
| SPARK-12645 | SparkR support hash function | Major | SparkR | Yanbo Liang | Yanbo Liang |
| SPARK-12788 | Simplify BooleanEquality by using casts | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-12761 | Clean up duplicated code in scala 2.11 repl.Main | Trivial | Spark Shell | Jakob Odersky | Jakob Odersky |
| SPARK-9383 | Merge script should reset back to previous ref instead of detached commit | Trivial | Project Infra | Josh Rosen | Josh Rosen |
| SPARK-12642 | improve the hash expression to be decoupled from unsafe row | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-12400 | Avoid writing a shuffle file if a partition has no output (empty) | Major | Shuffle | Reynold Xin | Saisai Shao |
| SPARK-2930 | clarify docs on using webhdfs with spark.yarn.access.namenodes | Trivial | Documentation, YARN | Thomas Graves | Thomas Graves |
| SPARK-12840 | Support passing arbitrary objects (not just expressions) into code generated classes | Major | SQL | Davies Liu | Davies Liu |
| SPARK-12856 | speed up hashCode of unsafe array | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-12860 | speed up safe projection for primitive types | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-12873 | Add more comment in HiveTypeCoercion for type widening | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-10985 | Avoid passing evicted blocks throughout BlockManager / CacheManager | Minor | Block Manager, Spark Core | Andrew Or | Josh Rosen |
| SPARK-12882 | simplify bucket tests and add more comments | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-12700 | SortMergeJoin and BroadcastHashJoin should support condition | Major | SQL | Davies Liu | Davies Liu |
| SPARK-11295 | Add packages to JUnit output for Python tests | Minor | PySpark, Tests | Gabor Liptak | Gabor Liptak |
| SPARK-9716 | BinaryClassificationEvaluator should accept Double prediction column | Minor | ML | Joseph K. Bradley | Benjamin Fradet |
| SPARK-12912 | Add test suite for EliminateSubQueries | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-12925 | Improve HiveInspectors.unwrap for StringObjectInspector.getPrimitiveWritableObject | Minor | SQL | Rajesh Balamohan | Rajesh Balamohan |
| SPARK-12898 | Consider having dummyCallSite for HiveTableScan | Major | SQL | Rajesh Balamohan | Rajesh Balamohan |
| SPARK-12616 | Union logical plan should support arbitrary number of children (rather than binary) | Major | SQL | Reynold Xin | Xiao Li |
| SPARK-12888 | benchmark the new hash expression | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-8968 | dynamic partitioning in spark sql performance issue due to the high GC overhead | Major | SQL | Fei Wang | Fei Wang |
| SPARK-12910 | Support for specifying version of R to use while creating sparkR libraries | Minor | SparkR | Shubhanshu Mishra | Shubhanshu Mishra |
| SPARK-12534 | Document missing command line options to Spark properties mapping | Minor | Deploy, Documentation, YARN | Felix Cheung | Felix Cheung |
| SPARK-12908 | Add tests to make sure that ml.classification.LogisticRegression returns meaningful result when labels are the same without intercept | Major | ML | DB Tsai | DB Tsai |
| SPARK-12872 | Support to specify the option for compression codec for JSON datasource. | Minor | SQL | Hyukjin Kwon | Hyukjin Kwon |
| SPARK-10498 | Add requirements file for create dev python tools | Minor | Build | holdenk | holdenk |
| SPARK-12120 | Improve exception message when failing to initialize HiveContext in PySpark | Minor | PySpark | Jeff Zhang | Jeff Zhang |
| SPARK-12932 | Bad error message with trying to create Dataset from RDD of Java objects that are not bean-compliant | Trivial | Java API | Andy Grove | Andy Grove |
| SPARK-12902 | Visualization and metrics for generated operators | Major | SQL | Davies Liu | Davies Liu |
| SPARK-1832 | Executor UI improvement suggestions | Major | Web UI | Thomas Graves | Alex Bozarth |
| SPARK-12975 | Throwing Exception when Bucketing Columns are part of Partitioning Columns | Major | SQL | Xiao Li | Xiao Li |
| SPARK-12905 | PCAModel return eigenvalues for PySpark | Minor | ML, PySpark | Yanbo Liang | Yanbo Liang |
| SPARK-12879 | improve unsafe row writing framework | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-12834 | Use type conversion instead of Ser/De of Pickle to transform JavaArray and JavaList | Major | PySpark | Xusen Yin | Xusen Yin |
| SPARK-10911 | Executors should System.exit on clean shutdown | Minor | Spark Core | Thomas Graves | Zhuo Liu |
| SPARK-8725 | Test modules in topological order in dev/run-tests and python/run-tests | Minor | Build, Project Infra | Josh Rosen | Josh Rosen |
| SPARK-10509 | Excessive param boiler plate code | Minor | ML, PySpark | holdenk | holdenk |
| SPARK-11622 | Make LibSVMRelation extends HadoopFsRelation and Add LibSVMOutputWriter | Minor | MLlib | Jeff Zhang | Jeff Zhang |
| SPARK-12903 | Add covar_samp and covar_pop for SparkR | Major | SparkR | Yanbo Liang | Yanbo Liang |
| SPARK-12967 | NettyRPC races with SparkContext.stop() and throws exception | Minor | Spark Core | Nishkam Ravi | Nishkam Ravi |
| SPARK-12926 | SQLContext to display warning message when non-sql configs are being set | Trivial | SQL | Tejas Patil | Tejas Patil |
| SPARK-13031 | Improve test coverage for whole stage codegen | Major | SQL | Davies Liu | Davies Liu |
| SPARK-12749 | Spark SQL JSON schema infernce should allow floating-point numbers as BigDecimal | Minor | SQL | Brandon Bradley | Brandon Bradley |
| SPARK-11955 | Mark one side fields in merging schema for safely pushdowning filters in parquet | Major | SQL | Liang-Chi Hsieh | Liang-Chi Hsieh |
| SPARK-13072 | simplify and improve murmur3 hash expression codegen | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-12656 | Rewrite Intersect phyiscal plan using semi-join | Major | SQL | Reynold Xin | Xiao Li |
| SPARK-12914 | Generate TungstenAggregate with grouping keys | Major | SQL | Davies Liu | Davies Liu |
| SPARK-13098 | remove GenericInternalRowWithSchema | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-6363 | Switch to Scala 2.11 for default build | Minor | Build | antonkulaga | Josh Rosen |
| SPARK-13100 | improving the performance of stringToDate method in DateTimeUtils.scala | Minor | SQL | Yang Wang | Yang Wang |
| SPARK-13070 | Points out which physical file is the trouble maker when Parquet schema merging fails | Minor | SQL | Cheng Lian | Cheng Lian |
| SPARK-13093 | improve null check in nullSafeCodeGen for unary, binary and ternary expression | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-12637 | Print stage info of finished stages properly | Trivial | Spark Core | Navis | Navis |
| SPARK-13130 | Make whole stage codegen variable names slightly easier to read | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-12362 | Create a full-fledged built-in SQL parser | Critical | SQL | Reynold Xin | Herman van Hovell |
| SPARK-13094 | No encoder implicits for Seq[Primitive] | Major | SQL | Deenar Toraskar | Michael Armbrust |
| SPARK-13138 | Add "logical" package prefix for ddl.scala | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-12913 | Reimplement stat functions as declarative function | Major | SQL | Davies Liu | Davies Liu |
| SPARK-13020 | fix random generator for map type | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-12951 | Support spilling in generate aggregate | Major | SQL | Davies Liu | Davies Liu |
| SPARK-13147 | improve readability of generated code | Major | SQL | Davies Liu | Davies Liu |
| SPARK-13131 | Use best time and average time in micro benchmark | Major | SQL | Davies Liu | Davies Liu |
| SPARK-13152 | Fix task metrics deprecation warning | Minor | Spark Core | holdenk | holdenk |
| SPARK-13113 | Remove unnecessary bit operation when decoding page number | Minor | Spark Core | Liang-Chi Hsieh | Liang-Chi Hsieh |
| SPARK-13168 | Collapse adjacent Repartition operations | Major | SQL | Josh Rosen | Josh Rosen |
| SPARK-13215 | Remove fallback in codegen | Major | SQL | Davies Liu | Davies Liu |
| SPARK-5865 | Add doc warnings for methods that return local data structures | Minor | Spark Core, SQL | Nicholas Chammas | Tommy Yu |
| SPARK-13132 | LogisticRegression spends 35% of its time fetching the standardization parameter | Minor | ML | Gary King | Gary King |
| SPARK-10963 | Make KafkaCluster api public | Minor | Streaming | Cody Koeninger | Cody Koeninger |
| SPARK-8964 | Use Exchange in limit operations (per partition limit -> exchange to one partition -> per partition limit) | Major | SQL | Josh Rosen | Josh Rosen |
| SPARK-12992 | Vectorize parquet decoding using ColumnarBatch | Major | SQL | Nong Li | Nong Li |
| SPARK-13086 | Spark Shell for 2.11 does not allow loading files via '-i' | Minor | Spark Shell | Iulian Dragos | Iulian Dragos |
| SPARK-13189 | Cleanup build references to Scala 2.10 | Minor | Build | Luciano Resende | Luciano Resende |
| SPARK-12950 | Improve performance of BytesToBytesMap | Major | SQL | Davies Liu | Davies Liu |
| SPARK-12476 | Implement JdbcRelation#unhandledFilters for removing unnecessary Spark Filter | Major | SQL | Takeshi Yamamuro | Takeshi Yamamuro |
| SPARK-11565 | replace deprecated DigestUtils.shaHex call | Minor | SQL | Gabor Liptak | Gabor Liptak |
| SPARK-13203 | Add scalastyle rule banning use of mutable.SynchronizedBuffer | Trivial | Build | Ted Yu | Ted Yu |
| SPARK-5095 | Support launching multiple mesos executors in coarse grained mesos mode | Major | Mesos | Timothy Chen | Timothy Chen |
| SPARK-13057 | Add benchmark codes and the performance results for implemented compression schemes for InMemoryRelation | Major | SQL | Takeshi Yamamuro | Takeshi Yamamuro |
| SPARK-13234 | Remove duplicated SQL metrics | Major | SQL | Davies Liu | Davies Liu |
| SPARK-13235 | Remove an Extra Distinct in Union | Major | SQL | Xiao Li | Xiao Li |
| SPARK-13264 | Remove multi-byte character in spark-env.sh.template | Trivial | Spark Core | Sasaki Toru | Sasaki Toru |
| SPARK-12915 | SQL metrics for generated operators | Major | SQL | Davies Liu | Davies Liu |
| SPARK-7889 | Jobs progress of apps on complete page of HistoryServer shows uncompleted | Minor | Spark Core | meiyoula | Steve Loughran |
| SPARK-6166 | Limit number of in flight outbound requests for shuffle fetch | Minor | Spark Core | Mridul Muralidharan | Sanket Reddy |
| SPARK-12974 | Add Python API for spark.ml bisecting k-means | Minor | ML, PySpark | Yanbo Liang | Yanbo Liang |
| SPARK-13154 | Add pydoc lint for docs | Trivial | PySpark | holdenk | holdenk |
| SPARK-12962 | PySpark support covar_samp and covar_pop | Minor | PySpark, SQL | Yanbo Liang | Yanbo Liang |
| SPARK-13293 | Generate code for Expand | Major | SQL | Davies Liu | Davies Liu |
| SPARK-13185 | Improve the performance of DateTimeUtils.StringToDate by reusing Calendar objects | Minor | SQL | Carson Wang | Carson Wang |
| SPARK-12503 | Pushdown a Limit on top of a Union | Major | Optimizer, SQL | Xiao Li | Josh Rosen |
| SPARK-13097 | Extend Binarizer to allow Double AND Vector inputs | Minor | ML | Mike Seddon | Mike Seddon |
| SPARK-12976 | Add LazilyGenerateOrdering and use it for RangePartitioner of Exchange. | Major | SQL | Takuya Ueshin | Takuya Ueshin |
| SPARK-13280 | FileBasedWriteAheadLog logger name should be under o.a.s namespace | Minor | Streaming | Marcelo Vanzin | Marcelo Vanzin |
| SPARK-13357 | Use generated projection and ordering for TakeOrderedAndProjectNode | Major | SQL | Takuya Ueshin | Takuya Ueshin |
| SPARK-13279 | Scheduler does O(N^2) operation when adding a new task set (making it prohibitively slow for scheduling 200K tasks) | Major | Scheduler, Spark Core | Sital Kedia | Sital Kedia |
| SPARK-12953 | RDDRelation write set mode will be better to avoid error "pair.parquet already exists" | Minor | Examples | shijinkui | shijinkui |
| SPARK-13324 | Update plugin, test, example dependencies for 2.x | Minor | Build, Examples, Spark Core | Sean Owen | Sean Owen |
| SPARK-13237 | Generate broadcast outer join | Major | SQL | Davies Liu | Davies Liu |
| SPARK-13339 | Clarify commutative / associative operator requirements for reduce, fold | Minor | Documentation | Sean Owen | Sean Owen |
| SPARK-12594 | Outer Join Elimination by Filter Condition | Critical | Optimizer, SQL | Xiao Li | Xiao Li |
| SPARK-13386 | ConnectedComponents should support maxIteration option | Minor | GraphX | zhengruifeng | zhengruifeng |
| SPARK-13414 | Add support for launching multiple Mesos dispatchers | Major | Mesos | Timothy Chen | Timothy Chen |
| SPARK-13416 | Add positive check for option 'numIter' in StronglyConnectedComponents | Trivial | GraphX | zhengruifeng | zhengruifeng |
| SPARK-13248 | Remove depecrated Streaming APIs | Minor | Streaming | Luciano Resende | Luciano Resende |
| SPARK-13136 | Data exchange (shuffle, broadcast) should only be handled by the exchange operator | Major | SQL | Reynold Xin | Herman van Hovell |
| SPARK-13271 | Better error message if 'path' is not specified | Major | SQL | Shixiong Zhu | Shixiong Zhu |
| SPARK-7617 | Word2VecModel fVector not normalized | Trivial | MLlib | Eric Li | YongGang Cao |
| SPARK-13399 | Investigate type erasure warnings in CheckpointSuite | Trivial | Tests | holdenk | holdenk |
| SPARK-10749 | Support multiple roles with Spark Mesos dispatcher | Major | Mesos | Timothy Chen | Timothy Chen |
| SPARK-13422 | Use HashedRelation instead of HashSet in Left Semi Joins | Minor | SQL | Herman van Hovell | Xiu (Joe) Guo |
| SPARK-13295 | ML/MLLIB: AFTSurvivalRegression: Improve AFTAggregator - Avoid creating new instances of arrays/vectors for each record | Major | ML, MLlib | Narine Kokhlikyan | Narine Kokhlikyan |
| SPARK-13429 | Unify Logistic Regression convergence tolerance of ML & MLlib | Minor | ML, MLlib | Yanbo Liang | Yanbo Liang |
| SPARK-13257 | Refine naive Bayes example code | Minor | Examples | Lenjoy Lin | Lenjoy Lin |
| SPARK-13364 | history server application column Id not sorting as number | Minor | Web UI | Thomas Graves | Zhuo Liu |
| SPARK-7729 | Executor which has been killed should also be displayed on Executors Tab. | Minor | Web UI | Archit Thakur | Lianhui Wang |
| SPARK-13329 | Considering output for statistics of logical plan | Major | SQL | Davies Liu | Davies Liu |
| SPARK-12759 | Spark should fail fast if --executor-memory is too small for spark to start | Trivial | Spark Shell | Imran Rashid | Daniel Jalova |
| SPARK-13467 | abstract python function to simplify pyspark code | Trivial | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-13376 | Improve column pruning | Major | SQL | Davies Liu | Davies Liu |
| SPARK-13490 | ML LinearRegression should cache standardization param value | Trivial | ML | Yanbo Liang | Yanbo Liang |
| SPARK-13387 | Add support for SPARK_DAEMON_JAVA_OPTS with MesosClusterDispatcher. | Minor | Mesos | Timothy Chen | Timothy Chen |
| SPARK-12757 | Use reference counting to prevent blocks from being evicted during reads | Major | Block Manager | Josh Rosen | Josh Rosen |
| SPARK-13361 | Add benchmark codes for Encoder#compress() in CompressionSchemeBenchmark | Major | SQL | Takeshi Yamamuro | Takeshi Yamamuro |
| SPARK-13503 | Support to specify the (writing) option for compression codec for TEXT | Minor | SQL | Hyukjin Kwon | Hyukjin Kwon |
| SPARK-12523 | Support long-running of the Spark On HBase and hive meta store. | Major | YARN | carlmartin | carlmartin |
| SPARK-13457 | Remove DataFrame RDD operations | Major | SQL | Cheng Lian | Cheng Lian |
| SPARK-13499 | Optimize vectorized parquet reader for dictionary encoded data and RLE decoding | Major | SQL | Nong Li | Nong Li |
| SPARK-13518 | Enable vectorized parquet reader by default | Major | SQL | Nong Li | Nong Li |
| SPARK-13530 | Add ShortType support to UnsafeRowParquetRecordReader | Major | SQL | Liang-Chi Hsieh | Liang-Chi Hsieh |
| SPARK-13526 | Refactor: Move SQLContext/HiveContext per-session state to separate class | Major | SQL | Andrew Or | Andrew Or |
| SPARK-13545 | Make MLlib LogisticRegressionWithLBFGS's default parameters consistent in Scala and Python | Minor | MLlib, PySpark | Yanbo Liang | Yanbo Liang |
| SPARK-12994 | It is not necessary to create ExecutorAllocationManager in local mode | Minor | Spark Core | Jeff Zhang | Jeff Zhang |
| SPARK-13481 | History server page with a default sorting as "desc" | Trivial | Spark Core | Zhuo Liu | Zhuo Liu |
| SPARK-13544 | Rewrite/Propagate constraints for Aliases in Aggregate | Major | SQL | Xiao Li | Xiao Li |
| SPARK-13213 | BroadcastNestedLoopJoin is very slow | Major | SQL | Davies Liu | Davies Liu |
| SPARK-13551 | Fix fix wrong comment and remove meanless lines in mllib.JavaBisectingKMeansExample | Trivial | MLlib | zhengruifeng | zhengruifeng |
| SPARK-13550 | Add java example for ml.clustering.BisectingKMeans | Trivial | ML | zhengruifeng | zhengruifeng |
| SPARK-13511 | Add wholestage codegen for limit | Major | SQL | Liang-Chi Hsieh | Liang-Chi Hsieh |
| SPARK-13582 | Improve performance of parquet reader with dictionary encoding | Major | SQL | Davies Liu | Davies Liu |
| SPARK-13008 | Make ML Python package all list have one algorithm per line | Trivial | ML, PySpark | Joseph K. Bradley | Joseph K. Bradley |
| SPARK-12817 | Remove CacheManager and replace it with new BlockManager.getOrElseUpdate method | Major | Block Manager | Josh Rosen | Josh Rosen |
| SPARK-13601 | Invoke task failure callbacks before calling outputstream.close() | Major | Spark Core | Davies Liu | Davies Liu |
| SPARK-13574 | Improve parquet dictionary decoding for strings | Minor | SQL | Nong Li | Nong Li |
| SPARK-13528 | Make the short names of compression codecs consistent in spark | Minor | Spark Core, SQL | Takeshi Yamamuro | Takeshi Yamamuro |
| SPARK-13617 | remove unnecessary GroupingAnalytics trait | Minor | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-13616 | Let SQLBuilder convert logical plan without a Project on top of it | Major | SQL | Liang-Chi Hsieh | Liang-Chi Hsieh |
| SPARK-13621 | TestExecutor.scala needs to be moved to test package | Trivial | Build, Tests | Devaraj K | Devaraj K |
| SPARK-13635 | Enable LimitPushdown optimizer rule because we have whole-stage codegen for Limit | Major | SQL | Liang-Chi Hsieh | Liang-Chi Hsieh |
| SPARK-13466 | Remove redundant project in colum pruning rule | Major | SQL | Liang-Chi Hsieh | Liang-Chi Hsieh |
| SPARK-13423 | Static analysis fixes for 2.x | Minor | ML, MLlib, Spark Core, SQL, Streaming | Sean Owen | Sean Owen |
| SPARK-13599 | Groovy-all ends up in spark-assembly if hive profile set | Minor | Build | Steve Loughran | Steve Loughran |
| SPARK-13647 | also check if numeric value is within allowed range in _verify_type | Major | PySpark, SQL | Wenchen Fan | Wenchen Fan |
| SPARK-13646 | QuantileDiscretizer counts dataset twice in getSampledInput | Trivial | ML | Abou Haydar Elias | Abou Haydar Elias |
| SPARK-13673 | script bin.cmd pollutes environment variables in Windows. | Minor | Windows | Masayoshi TSUZUKI | Masayoshi TSUZUKI |
| SPARK-13459 | Separate Alive and Dead Executors in Executor Totals Table | Minor | Web UI | Alex Bozarth | Alex Bozarth |
| SPARK-13694 | QueryPlan.expressions should always include all expressions | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-13404 | Create the variables for input when it's used | Major | SQL | Davies Liu | Davies Liu |
| SPARK-13659 | Remove returnValues from BlockStore APIs | Major | Block Manager | Josh Rosen | Josh Rosen |
| SPARK-13715 | Remove last usages of jblas in tests | Minor | Tests | Sean Owen | Sean Owen |
| SPARK-13695 | Don't cache MEMORY_AND_DISK blocks as bytes in memory store when reading spills | Major | Block Manager | Josh Rosen | Josh Rosen |
| SPARK-13740 | add null check for _verify_type in types.py | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-13593 | improve the createDataFrame method to accept data type string and verify the data |
Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-13668 | Reorder filter/join predicates to short-circuit isNotNull checks | Minor | SQL | Sameer Agarwal | Sameer Agarwal |
| SPARK-13625 | PySpark-ML method to get list of params for an obj should not check property attr | Major | ML, PySpark | Bryan Cutler | Bryan Cutler |
| SPARK-13702 | Use diamond operator for generic instance creation in Java code | Trivial | Examples | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-13763 | Remove Project when its projectList is Empty | Major | SQL | Xiao Li | Xiao Li |
| SPARK-13615 | GeneralizedLinearRegression support save/load | Major | ML | Yanbo Liang | Yanbo Liang |
| SPARK-13527 | Prune Filters based on Constraints | Major | Optimizer, SQL | Xiao Li | Xiao Li |
| SPARK-13775 | history server sort by completed time by default | Trivial | Web UI | Thomas Graves | Zhuo Liu |
| SPARK-13492 | Configure a custom webui_url for the Spark Mesos Framework | Minor | Mesos | Sergiusz Urbaniak | Sergiusz Urbaniak |
| SPARK-13766 | Inconsistent file extensions and omitted file extensions written by CSV, TEXT and JSON data sources | Minor | SQL | Hyukjin Kwon | Hyukjin Kwon |
| SPARK-13706 | Python Example for Train Validation Split Missing | Minor | ML, MLlib, PySpark | Jeremy | Jeremy |
| SPARK-13663 | Upgrade Snappy Java to <IP_ADDRESS> | Minor | Spark Core | Ted Yu | Y Y |
| SPARK-13758 | Error message is misleading when RDD refer to null spark context | Trivial | Documentation, Spark Core, Streaming | Mao, Wei | Mao, Wei |
| SPARK-13790 | Speed up ColumnVector's getDecimal | Minor | SQL | Nong Li | Nong Li |
| SPARK-13696 | Remove BlockStore interface to more cleanly reflect different memory and disk store responsibilities | Major | Block Manager | Josh Rosen | Josh Rosen |
| SPARK-3854 | Scala style: require spaces before { |
Major | Project Infra | Josh Rosen | Dongjoon Hyun |
| SPARK-13751 | Generate better code for Filter | Major | SQL | Davies Liu | Davies Liu |
| SPARK-13389 | SparkR support first/last with ignore NAs | Major | SparkR | Yanbo Liang | Yanbo Liang |
| SPARK-13732 | Remove projectList from Windows | Major | SQL | Xiao Li | Xiao Li |
| SPARK-13797 | Eliminate Unnecessary Window | Major | SQL | Xiao Li | Xiao Li |
| SPARK-13672 | Add python examples of BisectingKMeans in ML and MLLIB | Trivial | ML, MLlib | zhengruifeng | zhengruifeng |
| SPARK-13814 | Delete unnecessary imports in python examples files | Trivial | PySpark | zhengruifeng | zhengruifeng |
| SPARK-13810 | Add Port Configuration Suggestions on Bind Exceptions | Minor | Spark Core | Bjorn Jonsson | Bjorn Jonsson |
| SPARK-13834 | Update sbt and sbt plugins for 2.x. | Minor | . | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-13823 | Always specify Charset in String <-> byte[] conversions (and remaining Coverity items) | Minor | Spark Core, SQL, Streaming | Sean Owen | Sean Owen |
| SPARK-13139 | Create native DDL commands | Major | SQL | Reynold Xin | Andrew Or |
| SPARK-13833 | Guard against race condition when re-caching spilled bytes in memory | Major | Block Manager | Josh Rosen | Apache Spark |
| SPARK-13249 | Filter null keys for inner join | Major | SQL | Davies Liu | Sameer Agarwal |
| SPARK-11826 | Subtract BlockMatrix | Minor | MLlib | Ehsan Mohyedin Kermani | Ehsan Mohyedin Kermani |
| SPARK-13882 | Remove org.apache.spark.sql.execution.local | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-10380 | Confusing examples in pyspark SQL docs | Minor | Documentation, SQL | Michael Armbrust | Reynold Xin |
| SPARK-13661 | Avoid the copy of UnsafeRow in HashedRelation | Major | SQL | Davies Liu | Davies Liu |
| SPARK-13353 | Fast serialization for collecting DataFrame | Major | SQL | Davies Liu | Davies Liu |
| SPARK-13884 | Remove DescribeCommand's dependency on LogicalPlan | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-13890 | Remove some internal classes' dependency on SQLContext | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-13893 | Remove SQLContext.catalog (internal method) | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-12653 | Re-enable test "SPARK-8489: MissingRequirementError during reflection" | Minor | SQL | Reynold Xin | Dongjoon Hyun |
| SPARK-13281 | Switch broadcast of RDD to exception from warning | Trivial | Spark Core | holdenk | Wesley Tang |
| SPARK-13924 | officially support multi-insert | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-13816 | Add parameter checks for algorithms in Graphx | Trivial | GraphX | zhengruifeng | zhengruifeng |
| SPARK-13613 | Provide ignored tests to export test dataset into CSV format | Minor | ML | Yanbo Liang | Yanbo Liang |
| SPARK-13922 | Filter rows with null attributes in parquet vectorized reader | Major | SQL | Sameer Agarwal | Sameer Agarwal |
| SPARK-13869 | Remove redundant conditions while combining filters | Minor | SQL | Sameer Agarwal | Sameer Agarwal |
| SPARK-11011 | UserDefinedType serialization should be strongly typed | Minor | SQL | John Muller | Jakob Odersky |
| SPARK-13761 | Deprecate validateParams | Minor | ML | Joseph K. Bradley | yuhao yang |
| SPARK-13873 | Avoid the copy in whole stage codegen when there is no joins | Major | SQL | Davies Liu | Davies Liu |
| SPARK-13118 | Support for classes defined in package objects | Major | SQL | Michael Armbrust | Jakob Odersky |
| SPARK-13926 | Automatically use Kryo serializer when shuffling RDDs with simple types | Major | Shuffle | Josh Rosen | Josh Rosen |
| SPARK-13928 | Move org.apache.spark.Logging into org.apache.spark.internal.Logging | Major | Spark Core | Reynold Xin | Wenchen Fan |
| SPARK-13427 | Support USING clause in JOIN | Major | SQL | Dilip Biswal | Dilip Biswal |
| SPARK-13838 | Clear variable code to prevent it to be re-evaluated in BoundAttribute | Minor | SQL | Liang-Chi Hsieh | Liang-Chi Hsieh |
| SPARK-10788 | Decision Tree duplicates bins for unordered categorical features | Minor | ML | Joseph K. Bradley | Seth Hendrickson |
| SPARK-13974 | sub-query names do not need to be globally unique while generate SQL | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-13921 | Store serialized blocks as multiple chunks in MemoryStore | Major | Block Manager | Josh Rosen | Josh Rosen |
| SPARK-13930 | Apply fast serialization on collect limit | Major | SQL | Liang-Chi Hsieh | Liang-Chi Hsieh |
| SPARK-13972 | hive tests should fail if SQL generation failed | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-12182 | Distributed binning for trees in spark.ml | Minor | ML | Joseph K. Bradley | Seth Hendrickson |
| SPARK-14019 | Remove noop SortOrder in Sort | Major | SQL | Xiao Li | Xiao Li |
| SPARK-14028 | Remove deprecated methods; fix two other warnings | Minor | Streaming, Tests | Liwei Lin | Liwei Lin |
| SPARK-13986 | Remove DeveloperApi -annotation for non-publics |
Trivial | MLlib, Spark Core | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-14039 | make SubqueryHolder an inner class | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-14007 | Manage the memory for hash map for shuffle hash join | Major | SQL | Davies Liu | Davies Liu |
| SPARK-13805 | Direct consume ColumnVector in generated code when ColumnarBatch is used | Major | SQL | Kazuaki Ishizaki | Kazuaki Ishizaki |
| SPARK-13916 | For whole stage codegen, measure and add the execution duration as a metric | Minor | SQL | Nong Li | Nong Li |
| SPARK-13822 | Follow-ups of DataFrame/Dataset API unification | Blocker | Build, SQL | Cheng Lian | Cheng Lian |
| SPARK-13990 | Automatically pick serializer when caching RDDs | Major | Block Manager | Josh Rosen | Josh Rosen |
| SPARK-14058 | Incorrect docstring in Window.orderBy | Trivial | PySpark | Maciej Szymkiewicz | Maciej Szymkiewicz |
| SPARK-14038 | Enable native view by default | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-13953 | Support for specifying the field name for corrupted record at JSON datasource. | Minor | SQL | Hyukjin Kwon | Hyukjin Kwon |
| SPARK-14029 | Improve BooleanSimplification optimization by implementing Not canonicalization |
Minor | SQL | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-14072 | Show JVM information when we run Benchmark | Minor | . | Kazuaki Ishizaki | Kazuaki Ishizaki |
| SPARK-13401 | Fix SQL test warnings | Trivial | SQL, Tests | holdenk | Yong Tang |
| SPARK-12767 | Improve conditional expressions | Major | Optimizer, SQL | Reynold Xin | Reynold Xin |
| SPARK-13549 | Refactor the Optimizer Rule CollapseProject | Minor | SQL | Xiao Li | Xiao Li |
| SPARK-14075 | Refactor MemoryStore to be testable independent of BlockManager | Major | Block Manager, Spark Core | Josh Rosen | Josh Rosen |
| SPARK-13068 | Extend pyspark ml paramtype conversion | Trivial | ML, PySpark | holdenk | Seth Hendrickson |
| SPARK-14025 | Fix streaming job descriptions on the event line | Minor | Streaming, Web UI | Liwei Lin | Liwei Lin |
| SPARK-14085 | Star Expansion for Hash | Major | SQL | Xiao Li | Xiao Li |
| SPARK-12183 | Remove spark.mllib tree, forest implementations and use spark.ml | Minor | ML, MLlib | Joseph K. Bradley | Joseph K. Bradley |
| SPARK-10009 | PySpark Param of Vector type can be set with Python array or numpy.array | Major | ML, PySpark | Yanbo Liang | Seth Hendrickson |
| SPARK-14110 | PipedRDD to print the command ran on non zero exit | Trivial | Spark Core | Tejas Patil | Tejas Patil |
| SPARK-14030 | Add parameter check to several MLlib implementations | Minor | MLlib | zhengruifeng | zhengruifeng |
| SPARK-13980 | Incrementally serialize blocks while unrolling them in MemoryStore | Major | Block Manager | Josh Rosen | Josh Rosen |
| SPARK-14107 | PySpark spark.ml GBT algs need seed Param | Minor | ML, PySpark | Joseph K. Bradley | Seth Hendrickson |
| SPARK-13957 | Support group by ordinal in SQL | Major | SQL | Reynold Xin | Xiao Li |
| SPARK-14149 | Log exceptions in tryOrIOException | Major | Spark Core | Reynold Xin | Reynold Xin |
| SPARK-13887 | PyLint should fail fast to make errors easier to discover | Minor | Build, PySpark | holdenk | holdenk |
| SPARK-13919 | Resolving the Conflicts of ColumnPruning and PushPredicateThroughProject | Major | SQL | Xiao Li | Davies Liu |
| SPARK-14089 | Remove methods that has been deprecated since 1.1.x, 1.2.x and 1.3.x | Minor | MLlib, Spark Core | Liwei Lin | Liwei Lin |
| SPARK-13973 | ipython notebook is going away... |
Trivial | PySpark | Bogdan Pirvu | Rekha Joshi |
| SPARK-14175 | Simplify whole stage codegen interface | Major | SQL | Davies Liu | Davies Liu |
| SPARK-14161 | Parse Drop Database DDL command | Major | SQL | Xiao Li | Xiao Li |
| SPARK-14157 | Parse Drop Function DDL command | Major | SQL | Liang-Chi Hsieh | Liang-Chi Hsieh |
| SPARK-14177 | Parse DDL command: "DESCRIBE DATABASE" and "ALTER DATABASE SET DBPROPERTIES" | Major | SQL | Xiao Li | Xiao Li |
| SPARK-10691 | Make Logistic, Linear Regression Model evaluate() method public | Major | ML | Hao Ren | Joseph K. Bradley |
| SPARK-12494 | Array out of bound Exception in KMeans Yarn Mode | Trivial | MLlib | Anandraj | Sean Owen |
| SPARK-14102 | Block reset command in SparkShell |
Minor | Spark Shell | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-13742 | Add non-iterator interface to RandomSampler | Major | Spark Core | Liang-Chi Hsieh | Liang-Chi Hsieh |
| SPARK-13844 | Generate better code for filters with a non-nullable column | Major | SQL | Kazuaki Ishizaki | Kazuaki Ishizaki |
| SPARK-14052 | Build BytesToBytesMap in HashedRelation | Major | SQL | Davies Liu | Davies Liu |
| SPARK-14202 | python_full_outer_join should use generator expression instead of list comp | Major | PySpark | Maciej Szymkiewicz | Maciej Szymkiewicz |
| SPARK-14155 | Hide UserDefinedType in Spark 2.0 | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-14169 | Add UninterruptibleThread | Major | Spark Core | Shixiong Zhu | Shixiong Zhu |
| SPARK-14205 | remove trait Queryable | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-13981 | Improve Filter generated code to defer variable evaluation within operator | Minor | SQL | Nong Li | Nong Li |
| SPARK-14210 | Add timing metric for how long the query spent in scan | Minor | . | Nong Li | Nong Li |
| SPARK-14095 | LogisticRegression fails when a DataFrame has only a one-class label | Minor | ML, MLlib | Grzegorz Chilkiewicz | Feynman Liang |
| SPARK-14227 | Add method for printing out generated code for debugging | Minor | SQL | Eric Liang | Eric Liang |
| SPARK-14215 | Support chained Python UDF | Major | SQL | Davies Liu | Davies Liu |
| SPARK-14225 | Cap the length of toCommentSafeString at 128 chars | Major | SQL | Reynold Xin | Sameer Agarwal |
| SPARK-14254 | Add logs to help investigate the network performance | Major | Spark Core | Shixiong Zhu | Shixiong Zhu |
| SPARK-14268 | rename toRowExpressions and fromRowExpression to serializer and deserializer in ExpressionEncoder | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-14181 | TrainValidationSplit should have HasSeed | Minor | ML | Xusen Yin | Xusen Yin |
| SPARK-14259 | Add config to control maximum number of files when coalescing partitions | Minor | SQL | Nong Li | Takeshi Yamamuro |
| SPARK-14164 | Improve input layer validation of MultilayerPerceptronClassifier | Minor | MLlib | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-14062 | Put metrics.properties to distributed cache for Spark running on Yarn. Also Fix log4j propagation issue | Minor | YARN | Saisai Shao | Saisai Shao |
| SPARK-14069 | Improve SparkStatusTracker to also track executor information | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-14304 | Fix tests that don't create temp files in the java.io.tmpdir folder |
Minor | Tests | Shixiong Zhu | Shixiong Zhu |
| SPARK-14281 | Fix the java8-tests profile and run those tests in Jenkins | Major | Project Infra, Tests | Josh Rosen | Josh Rosen |
| SPARK-14277 | Significant amount of CPU is being consumed in SnappyNative arrayCopy method | Major | Shuffle | Sital Kedia | Sital Kedia |
| SPARK-14267 | Execute multiple Python UDFs in single batch | Major | SQL | Davies Liu | Davies Liu |
| SPARK-14242 | avoid too many copies in network when a network frame is large | Major | Input/Output, Spark Core | Zhang, Liye | Zhang, Liye |
| SPARK-11262 | Unit test for gradient, loss layers, memory management for multilayer perceptron | Major | ML, Tests | Alexander Ulanov | Alexander Ulanov |
| SPARK-14303 | Refactor k-means code in SparkRWrappers | Major | ML, SparkR | Xiangrui Meng | Yanbo Liang |
| SPARK-14070 | Use ORC data source for SQL queries on ORC tables | Minor | SQL | Tejas Patil | Tejas Patil |
| SPARK-13674 | Add wholestage codegen support to Sample | Major | SQL | Liang-Chi Hsieh | Liang-Chi Hsieh |
| SPARK-13825 | Upgrade to Scala 2.11.8 | Minor | Spark Core | Jacek Laskowski | Jacek Laskowski |
| SPARK-12857 | Streaming tab in web UI uses records and events interchangeably | Minor | Documentation, Streaming, Web UI | Jacek Laskowski | Liwei Lin |
| SPARK-13241 | add long--formatted timestamps to org.apache.spark.status.api.v1.ApplicationAttemptInfo | Minor | Web UI | Steve Loughran | Alex Bozarth |
| SPARK-14251 | Add SQL command for printing out generated code for debugging | Major | SQL | Reynold Xin | Dongjoon Hyun |
| SPARK-14338 | Improve SimplifyConditionals rule to handle null in IF/CASEWHEN |
Major | Optimizer, SQL | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-14056 | Add s3 configurations and spark.hadoop.* configurations to hive configuration | Minor | EC2, SQL | Sital Kedia | Sital Kedia |
| SPARK-13996 | Add more not null attributes for Filter codegen | Major | SQL | Liang-Chi Hsieh | Liang-Chi Hsieh |
| SPARK-12425 | DStream union optimisation | Minor | Streaming | Guillaume Poulin | Guillaume Poulin |
| SPARK-14386 | spark.ml DecisionTreeModel abstraction should not be exposed | Major | ML | Joseph K. Bradley | Joseph K. Bradley |
| SPARK-14284 | Rename KMeansSummary.size to clusterSizes | Trivial | ML | Joseph K. Bradley | Shally Sangal |
| SPARK-14345 | Decouple deserializer expression resolution from ObjectOperator | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-13063 | Make the SPARK YARN STAGING DIR as configurable | Minor | YARN | Devaraj K | Devaraj K |
| SPARK-529 | Have a single file that controls the environmental variables and spark config options | Major | Spark Core | Reynold Xin | Marcelo Vanzin |
| SPARK-13211 | StreamingContext throws NoSuchElementException when created from non-existent checkpoint directory | Minor | Streaming | Jacek Laskowski | Sean Owen |
| SPARK-14296 | whole stage codegen support for Dataset.map | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-14416 | Add thread-safe comments for CoarseGrainedSchedulerBackend's fields | Minor | . | Shixiong Zhu | Shixiong Zhu |
| SPARK-14426 | Merge ParserUtils and ParseUtils | Major | SQL | Kousuke Saruta | Kousuke Saruta |
| SPARK-13538 | Add GaussianMixture to ML | Minor | ML | zhengruifeng | zhengruifeng |
| SPARK-14317 | Clean up hash join | Major | . | Davies Liu | Davies Liu |
| SPARK-13430 | Expose ml summary function in PySpark for classification and regression models | Major | ML, MLlib, PySpark | Shubhanshu Mishra | Bryan Cutler |
| SPARK-14444 | Add a new scalastyle NoScalaDoc to prevent ScalaDoc-style multiline comments |
Major | Build | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-14290 | Fully utilize the network bandwidth for Netty RPC by avoid significant underlying memory copy | Major | Input/Output, Spark Core | Zhang, Liye | Zhang, Liye |
| SPARK-14245 | webUI should display the user | Major | Web UI | Thomas Graves | Alex Bozarth |
| SPARK-12384 | Allow -Xms to be set differently then -Xmx | Minor | Spark Submit, YARN | Thomas Graves | Dhruve Ashar |
| SPARK-12740 | grouping()/grouping_id() should work with having and order by | Major | SQL | Davies Liu | Davies Liu |
| SPARK-14410 | SessionCatalog needs to check function existence | Major | SQL | Yin Huai | Andrew Or |
| SPARK-14270 | whole stage codegen support for typed filter | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-14470 | Allow for overriding both httpclient and httpcore versions | Trivial | Spark Core | Aaron Tokhy | Aaron Tokhy |
| SPARK-14448 | Improvements to ColumnVector | Minor | SQL | Ted Yu | Ted Yu |
| SPARK-14435 | Shade Kryo in our custom Hive 1.2.1 fork | Major | Build | Josh Rosen | Josh Rosen |
| SPARK-14482 | Change default compression codec for Parquet from gzip to snappy | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-14419 | Improve the HashedRelation for key fit within Long | Major | SQL | Davies Liu | Davies Liu |
| SPARK-14339 | Add python examples for DCT,MinMaxScaler,MaxAbsScaler | Minor | . | zhengruifeng | zhengruifeng |
| SPARK-14362 | DDL Native Support: Drop View | Major | SQL | Xiao Li | Xiao Li |
| SPARK-13687 | Cleanup pyspark temporary files | Minor | PySpark | Damir | holdenk |
| SPARK-14497 | Use top instead of sortBy() to get top N frequent words as dict in CountVectorizer | Major | ML | Feng Wang | Feng Wang |
| SPARK-14415 | All functions should show usages by command DESC FUNCTION |
Major | SQL | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-14372 | Dataset.randomSplit() needs a Java version | Major | SQL | Cheng Lian | Rekha Joshi |
| SPARK-14510 | Add args-checking for LDA and StreamingKMeans | Minor | MLlib | zhengruifeng | zhengruifeng |
| SPARK-14502 | Add optimization for Binary Comparison Simplification | Major | SQL | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-10521 | Utilize Docker to test DB2 JDBC Dialect support | Major | SQL | Luciano Resende | Luciano Resende |
| SPARK-14475 | Propagate user-defined context from driver to executors | Major | Spark Core | Eric Liang | Eric Liang |
| SPARK-14508 | Add a new ScalaStyle Rule OmitBracesInCase |
Major | . | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-3724 | RandomForest: More options for feature subset size | Minor | MLlib | Joseph K. Bradley | Yong Tang |
| SPARK-14324 | Refactor GLMs code in SparkRWrappers | Major | ML, SparkR | Xiangrui Meng | Yanbo Liang |
| SPARK-13322 | AFTSurvivalRegression should support feature standardization | Major | ML | Yanbo Liang | Yanbo Liang |
| SPARK-14556 | Code clean-ups for package o.a.s.sql.execution.streaming.state | Minor | SQL | Liwei Lin | Liwei Lin |
| SPARK-14562 | Improve constraints propagation in Union | Major | SQL | Davies Liu | Davies Liu |
| SPARK-14547 | Avoid DNS resolution for reusing connections | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-14581 | Improve filter push down | Major | SQL | Davies Liu | Davies Liu |
| SPARK-14375 | Unit test for spark.ml KMeansSummary | Major | ML | Joseph K. Bradley | Yanbo Liang |
| SPARK-14509 | Add python CountVectorizerExample | Minor | Documentation, ML, PySpark | zhengruifeng | zhengruifeng |
| SPARK-14472 | Cleanup PySpark-ML Java wrapper classes so that JavaWrapper will inherit from JavaCallable | Minor | ML, PySpark | Bryan Cutler | Bryan Cutler |
| SPARK-14596 | Remove not used SqlNewHadoopRDD | Trivial | SQL | Hyukjin Kwon | Hyukjin Kwon |
| SPARK-14518 | Support Comment in CREATE VIEW | Major | SQL | Xiao Li | Xiao Li |
| SPARK-14565 | RandomForest should use parseInt and parseDouble for feature subset size instead of regexes | Minor | ML | Xiangrui Meng | Yong Tang |
| SPARK-12869 | Optimize conversion from BlockMatrix to IndexedRowMatrix | Minor | MLlib | Fokko Driesprong | Fokko Driesprong |
| SPARK-14601 | Minor doc/usage changes related to removal of Spark assembly | Major | Documentation | Mark Grover | Mark Grover |
| SPARK-14275 | Reimplement TypedAggregateExpression to DeclarativeAggregate | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-14630 | Code style: public abstract methods should have explicit return types | Minor | Build | Liwei Lin | Liwei Lin |
| SPARK-14370 | Avoid creating duplicate ids in OnlineLDAOptimizer | Trivial | MLlib | Joseph K. Bradley | Pravin Gadakh |
| SPARK-14633 | Use more readable format to show memory bytes in Error Message | Trivial | . | Peter Ableda | Peter Ableda |
| SPARK-14104 | All Python param setters should use the \_set method. |
Minor | ML, PySpark | Seth Hendrickson | Seth Hendrickson |
| SPARK-14605 | Python spark.ml classes should use unicode uid | Minor | ML, PySpark | Joseph K. Bradley | Joseph K. Bradley |
| SPARK-14696 | Needs implicit encoders for boxed primitive types | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-13807 | De-duplicate Python\*Helper instantiation code in PySpark streaming |
Major | PySpark, Streaming | Josh Rosen | Josh Rosen |
| SPARK-14614 | Add bround function |
Major | SQL | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-14655 | Add assert\_true function |
Major | SQL | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-14440 | Remove PySpark ml.pipeline's specific Reader and Writer | Trivial | ML, PySpark | Xusen Yin | Xusen Yin |
| SPARK-14710 | Rename gen/genCode to genCode/doGenCode to better reflect the semantics | Major | SQL | Sameer Agarwal | Sameer Agarwal |
| SPARK-14718 | Avoid mutating ExprCode in doGenCode | Major | . | Sameer Agarwal | Sameer Agarwal |
| SPARK-14722 | Rename upstreams() -> inputRDDs() in WholeStageCodegen | Major | SQL | Sameer Agarwal | Sameer Agarwal |
| SPARK-13904 | Add support for pluggable cluster manager | Major | Scheduler | Hemant Bhanawat | Hemant Bhanawat |
| SPARK-14577 | spark.sql.codegen.maxCaseBranches config option | Major | SQL | Reynold Xin | Dongjoon Hyun |
| SPARK-14491 | refactor object operator framework to make it easy to eliminate serializations | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-4226 | SparkSQL - Add support for subqueries in predicates | Major | SQL | Terry Siu | Herman van Hovell |
| SPARK-14733 | Allow custom timing control in microbenchmarks | Major | . | Eric Liang | Eric Liang |
| SPARK-14717 | Scala, Python APIs for Dataset.unpersist differ in default blocking value | Minor | PySpark, SQL | Joseph K. Bradley | Felix Cheung |
| SPARK-13905 | Change signature of as.data.frame() to be consistent with the R base package | Major | SparkR | Sun Rui | Sun Rui |
| SPARK-14600 | Push predicates through Expand | Major | SQL | Davies Liu | Wenchen Fan |
| SPARK-14639 | Add bround function in Python/R. |
Major | PySpark, SparkR | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-9013 | generate MutableProjection directly instead of return a function | Minor | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-14687 | Call path.getFileSystem(conf) instead of call FileSystem.get(conf) | Minor | MLlib, Spark Core, SQL | Liwei Lin | Liwei Lin |
| SPARK-14214 | Update State Store to give a more get/put hashmap-style interface | Major | SQL | Tathagata Das | Tathagata Das |
| SPARK-13842 | Consider __iter__ and __getitem__ methods for pyspark.sql.types.StructType | Minor | PySpark | Shea Parkes | Shea Parkes |
| SPARK-8393 | JavaStreamingContext#awaitTermination() throws non-declared InterruptedException | Minor | Streaming | Jaromir Vanek | Sean Owen |
| SPARK-14787 | Upgrade Joda-Time library from 2.9 to 2.9.3 | Trivial | SQL | Hyukjin Kwon | Hyukjin Kwon |
| SPARK-13988 | Large history files block new applications from showing up in History UI. | Major | Spark Core | Parth Brahmbhatt | Parth Brahmbhatt |
| SPARK-14780 | Add setLogLevel to SparkR |
Major | SparkR | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-14724 | Improve performance of sorting by using radix sort when possible | Major | Spark Core | Eric Liang | Eric Liang |
| SPARK-7992 | Hide private classes/objects in in generated Java API doc | Major | Build, Documentation | Xiangrui Meng | Jakob Odersky |
| SPARK-14479 | GLM supports output link prediction | Major | ML, SparkR | Yanbo Liang | Yanbo Liang |
| SPARK-10001 | Allow Ctrl-C in spark-shell to kill running job | Minor | Spark Shell | Cheolsoo Park | Jakob Odersky |
| SPARK-6429 | Add to style checker "hashCode and equals should be defined together" | Major | Build | Reynold Xin | Joan Goyeau |
| SPARK-14669 | Some SQL metrics is broken when whole-stage codegen enabled | Major | SQL | Davies Liu | Davies Liu |
| SPARK-13348 | Avoid duplicated broadcasts | Major | SQL | Davies Liu | Davies Liu |
| SPARK-14796 | Add spark.sql.optimizer.inSetConversionThreshold config option | Major | Optimizer | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-14582 | Increase the parallelism for small tables | Major | SQL | Davies Liu | Davies Liu |
| SPARK-14855 | Add "Exec" suffix to all physical operators | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-14863 | Cache TreeNode's hashCode | Major | SQL | Josh Rosen | Josh Rosen |
| SPARK-14866 | Break SQLQuerySuite out into smaller test suites | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-14551 | Reduce number of NameNode calls in OrcRelation with FileSourceStrategy mode | Minor | SQL | Rajesh Balamohan | Rajesh Balamohan |
| SPARK-14594 | Improve error messages for RDD API | Major | SparkR | Marco Gaido | Felix Cheung |
| SPARK-14871 | Disable StatsReportListener to declutter output | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-14869 | Don't mask exceptions in ResolveRelations | Major | . | Reynold Xin | Reynold Xin |
| SPARK-13267 | Document ?params for the v1 REST API | Minor | Documentation | Steve Loughran | Steve Loughran |
| SPARK-14548 | Support !> and !< operator in Spark SQL | Minor | SQL | Jia Li | Jia Li |
| SPARK-14790 | Scalastyle should run on compile in sbt | Major | . | Eric Liang | Eric Liang |
| SPARK-14758 | Add checking for StepSize and Tol | Trivial | ML | zhengruifeng | zhengruifeng |
| SPARK-14636 | Spark should fail fast if executor/driver memory is too small for the StaticMemoryManager | Minor | . | Peter Ableda | Peter Ableda |
| SPARK-14768 | Remove expectedType arg for PySpark Param | Minor | ML, PySpark | Joseph K. Bradley | Jason C Lee |
| SPARK-14889 | scala.MatchError: NONE (of class scala.Enumeration$Val) when spark.scheduler.mode=NONE | Minor | Spark Core | Jacek Laskowski | Subhobrata Dey |
| SPARK-14756 | Use parseLong instead of valueOf | Trivial | . | Azeem Jiva | Azeem Jiva |
| SPARK-14912 | Propagate data source options to Hadoop configurations | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-11559 | Make runs no effect in k-means |
Major | MLlib | Xiangrui Meng | Yanbo Liang |
| SPARK-14903 | Revert: Change MLWritable.write to be a property | Major | ML, PySpark | Joseph K. Bradley | Joseph K. Bradley |
| SPARK-14853 | Support LeftSemi/LeftAnti in SortMergeJoin | Major | SQL | Davies Liu | Davies Liu |
| SPARK-14907 | Use repartition in GLMRegressionModel.save | Trivial | MLlib | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-14514 | Add python example for VectorSlicer | Minor | Documentation, ML, PySpark | zhengruifeng | zhengruifeng |
| SPARK-12301 | Remove final from classes in spark.ml trees and ensembles where possible | Major | ML | Joseph K. Bradley | Joseph K. Bradley |
| SPARK-14874 | Remove the obsolete Batch representation | Minor | SQL | Liwei Lin | Liwei Lin |
| SPARK-14729 | Implement an existing cluster manager with New ExternalClusterManager interface | Minor | Scheduler | Hemant Bhanawat | Hemant Bhanawat |
| SPARK-14867 | Remove --force option in build/mvn . |
Minor | Build | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-14899 | Remove spark.ml HashingTF hashingAlg option | Major | ML, PySpark | Joseph K. Bradley | Yanbo Liang |
| SPARK-14966 | SizeEstimator should ignore classes in the scala.reflect package | Major | Spark Core | Josh Rosen | Josh Rosen |
| SPARK-14961 | Support LongToUnsafeRowMap larger than 1G | Major | SQL | Davies Liu | Davies Liu |
| SPARK-14626 | Simplify accumulators and task metrics | Major | Spark Core | Reynold Xin | Reynold Xin |
| SPARK-14576 | Spark console should display Web UI url | Minor | Web UI | Ergin Seyfe | Ergin Seyfe |
| SPARK-12235 | Enhance mutate() to support replace existing columns | Major | SparkR | Sun Rui | Sun Rui |
| SPARK-6735 | Provide options to make maximum executor failure count ( which kills the application ) relative to a window duration or disable it. | Major | Spark Submit, YARN | Twinkle Sachdeva | Saisai Shao |
| SPARK-14882 | Clarify that Spark can be cross-built for other Scala versions | Trivial | Documentation | Ben McCann | Sean Owen |
| SPARK-14916 | A more friendly tostring for FreqItemset in mllib.fpm | Trivial | MLlib | yuhao yang | yuhao yang |
| SPARK-12810 | PySpark CrossValidatorModel should support avgMetrics | Major | ML, PySpark | Feynman Liang | Kai Jiang |
| SPARK-14862 | Tree and ensemble classification: do not require label metadata | Major | ML | Joseph K. Bradley | Joseph K. Bradley |
| SPARK-14836 | Zip local jars before uploading to distributed cache | Minor | YARN | Saisai Shao | Saisai Shao |
| SPARK-14829 | Deprecate GLM APIs using SGD | Major | MLlib | Joseph K. Bradley | zhengruifeng |
| SPARK-12660 | Rewrite except using anti-join | Major | SQL | Reynold Xin | Xiao Li |
| SPARK-14996 | Add TPCDS Benchmark Queries for SparkSQL | Major | SQL | Sameer Agarwal | Sameer Agarwal |
| SPARK-14969 | Remove unnecessary compute function in LogisticGradient | Trivial | MLlib | ding | ding |
| SPARK-13664 | Simplify and Speedup HadoopFSRelation | Blocker | SQL | Michael Armbrust | Michael Armbrust |
| SPARK-13785 | Deprecate model field in ML model summary classes | Minor | ML | Joseph K. Bradley | Joseph K. Bradley |
| SPARK-12854 | Vectorize Parquet reader | Major | SQL | Reynold Xin | Nong Li |
| SPARK-14646 | k-means save/load should put one cluster per row | Minor | ML | Xiangrui Meng | Joseph K. Bradley |
| SPARK-14858 | Push predicates with subquery | Major | SQL | Davies Liu | Herman van Hovell |
| SPARK-15003 | Use ConcurrentHashMap in place of HashMap for NewAccumulator.originals | Minor | Spark Core | Ted Yu | Ted Yu |
| SPARK-14850 | VectorUDT/MatrixUDT should take primitive arrays without boxing | Critical | ML, SQL | Xiangrui Meng | Wenchen Fan |
| SPARK-14825 | Merge functionality in Hive module into SQL core module | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-14831 | Make ML APIs in SparkR consistent | Critical | ML, SparkR | Xiangrui Meng | Timothy Hunter |
| SPARK-13485 | (Dataset-oriented) API evolution in Spark 2.0 | Blocker | SQL | Reynold Xin | Reynold Xin |
| SPARK-14952 | Remove methods that were deprecated in 1.6.0 | Minor | ML, Spark Core | Herman van Hovell | Herman van Hovell |
| SPARK-14060 | Move StringToColumn implicit class into SQLImplicits | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-14637 | object expressions cleanup | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-12454 | Add ExpressionDescription to expressions that are registered in FunctionRegistry | Major | SQL | Yin Huai | Dongjoon Hyun |
| SPARK-14830 | Add RemoveRepetitionFromGroupExpressions optimizer | Major | Optimizer | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-13745 | Support columnar in memory representation on Big Endian platforms | Major | SQL | Tim Preece | Pete Robbins |
| SPARK-10343 | Consider nullability of expression in codegen | Critical | SQL | Davies Liu | Davies Liu |
| SPARK-14747 | Add assertStreaming/assertNoneStreaming checks in DataFrameWriter | Minor | SQL | Liwei Lin | Liwei Lin |
| SPARK-15047 | Cleanup SQLParser | Minor | SQL | Herman van Hovell | Herman van Hovell |
| SPARK-14685 | Properly document heritability of localProperties | Minor | Spark Core | Marcin Tustin | Marcin Tustin |
| SPARK-13749 | Faster pivot implementation for many distinct values with two phase aggregation | Major | SQL | Andrew Ray | Andrew Ray |
| SPARK-6717 | Clear shuffle files after checkpointing in ALS | Major | MLlib | Xiangrui Meng | holdenk |
| SPARK-14971 | PySpark ML Params setter code clean up | Minor | ML, PySpark | Yanbo Liang | Yanbo Liang |
| SPARK-15091 | Fix warnings and a failure in SparkR test cases with testthat version 1.0.1 | Major | SparkR | Sun Rui | Sun Rui |
| SPARK-9819 | reduceBy(KeyAnd)Window should specify which is the accumulator argument in invReduceFunc | Minor | Documentation, Streaming | François Garillot | François Garillot |
| SPARK-15104 | Bad spacing in log line | Trivial | . | Andrew Ash | Andrew Ash |
| SPARK-14422 | Improve handling of optional configs in SQLConf | Minor | SQL | Marcelo Vanzin | Sandeep Singh |
| SPARK-13269 | Expose more executor stats in stable status API | Major | Spark Core | Andrew Or | Wenchen Fan |
| SPARK-14844 | KMeansModel in spark.ml should allow to change featureCol and predictionCol | Trivial | ML | Dominik Jastrzębski | Dominik Jastrzębski |
| SPARK-4224 | Support group acls | Major | Spark Core | Thomas Graves | Dhruve Ashar |
| SPARK-14951 | Subexpression elimination in wholestage codegen version of TungstenAggregate | Major | SQL | Liang-Chi Hsieh | Liang-Chi Hsieh |
| SPARK-15121 | Improve logging of external shuffle handler | Minor | Shuffle | Thomas Graves | Thomas Graves |
| SPARK-13001 | Coarse-grained Mesos scheduler should reject offers for longer period of time when reached max cores | Major | Mesos, Scheduler | Sebastien Rainville | Sebastien Rainville |
| SPARK-15132 | Debug log for generated code should be printed with proper indentation | Trivial | SQL | Kousuke Saruta | Kousuke Saruta |
| SPARK-15045 | Remove dead code in TaskMemoryManager.cleanUpAllAllocatedMemory for pageTable | Minor | Spark Core | Jacek Laskowski | Abhinav Gupta |
| SPARK-15123 | upgrade org.json4s to 3.2.11 version | Minor | Spark Core | Lining Sun | Lining Sun |
| SPARK-15106 | Add package documentation for ML and remove BETA from Scala & Java for ML pipeline API. | Trivial | Documentation, ML, PySpark | holdenk | holdenk |
| SPARK-15158 | Too aggressive logging in SizeBasedRollingPolicy? | Trivial | Spark Core | Kai Wang | Kai Wang |
| SPARK-9926 | Parallelize file listing for partitioned Hive table | Major | SQL | Cheolsoo Park | Ryan Blue |
| SPARK-15152 | Scaladoc and Code style Improvements | Minor | Documentation, ML, Spark Core, SQL, YARN | Jacek Laskowski | Jacek Laskowski |
| SPARK-14512 | Add python example for QuantileDiscretizer | Minor | . | zhengruifeng | zhengruifeng |
| SPARK-14050 | Add multiple languages support for Stop Words Remover | Major | ML | Burak KÖSE | Burak KÖSE |
| SPARK-1239 | Improve fetching of map output statuses | Major | Shuffle, Spark Core | Patrick Wendell | Thomas Graves |
| SPARK-15178 | Remove LazyFileRegion | Major | Shuffle | Thomas Graves | Sandeep Singh |
| SPARK-15136 | Linkify ML PyDoc | Minor | . | holdenk | holdenk |
| SPARK-15172 | Warning message should explicitly tell user initial coefficients is ignored if its size doesn't match expected size in LogisticRegression | Trivial | ML | ding | ding |
| SPARK-15093 | create/delete/rename directory for InMemoryCatalog operations if needed | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-15225 | Replace SQLContext with SparkSession in Encoder documentation | Minor | Documentation | Liang-Chi Hsieh | Liang-Chi Hsieh |
| SPARK-15220 | Add hyperlink to "running application" and "completed application" | Minor | Web UI | Mao, Wei | Mao, Wei |
| SPARK-15210 | Add missing @DeveloperApi annotation in sql.types | Trivial | SQL | zhengruifeng | zhengruifeng |
| SPARK-14972 | Improve performance of JSON schema inference's inferField step | Major | SQL | Josh Rosen | Josh Rosen |
| SPARK-15229 | Make case sensitivity setting internal | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-14542 | PipeRDD should allow configurable buffer size for the stdin writer | Minor | . | Sital Kedia | Sital Kedia |
| SPARK-14773 | Enable the tests in HiveCompatibilitySuite for subquery | Major | SQL | Davies Liu | Herman van Hovell |
| SPARK-15195 | Improve PyDoc for ml.tuning | Trivial | Documentation, PySpark | holdenk | holdenk |
| SPARK-15249 | Use FunctionResource instead of (String, String) in CreateFunction and CatalogFunction for resource | Minor | SQL | Sandeep Singh | Sandeep Singh |
| SPARK-15250 | Remove deprecated json API in DataFrameReader | Minor | SQL | Hyukjin Kwon | Hyukjin Kwon |
| SPARK-15255 | RDD name from DataFrame op should not include full local relation data | Minor | SQL, Web UI | Joseph K. Bradley | Davies Liu |
| SPARK-14340 | Add Scala Example and User DOC for ml.BisectingKMeans | Minor | . | zhengruifeng | zhengruifeng |
| SPARK-15189 | ml.Evaluation pydoc issues | Trivial | Documentation, ML, PySpark | holdenk | holdenk |
| SPARK-14976 | make StreamingContext.textFileStream support wildcard | Minor | Streaming | Mao, Wei | Mao, Wei |
| SPARK-15238 | Clarify Python 3 support in docs | Trivial | Documentation, PySpark | Nicholas Chammas | Nicholas Chammas |
| SPARK-15160 | support data source table in InMemoryCatalog | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-15281 | PySpark ML GBTRegressor lacks impurity param | Trivial | ML, PySpark | holdenk | holdenk |
| SPARK-15188 | PySpark NaiveBayes is missing Thresholds param | Trivial | ML, PySpark | holdenk | holdenk |
| SPARK-15310 | Rename HiveTypeCoercion -> TypeCoercion | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-15061 | Upgrade Py4J to 0.10.1 | Minor | PySpark | Chris Kanich | holdenk |
| SPARK-15267 | Refactor and add some classes for options in datasources like CSVOptions or JSONOptions | Minor | SQL | Hyukjin Kwon | Hyukjin Kwon |
| SPARK-15197 | Improve documentation for countApprox and related functions | Minor | . | Nicholas Tietz | Nicholas Tietz |
| SPARK-14979 | Add examples for GeneralizedLinearRegression | Minor | ML, PySpark | Yanbo Liang | Yanbo Liang |
| SPARK-15290 | Move annotations, like @Since / @DeveloperApi, into spark-tags | Minor | Build | Sean Owen | Sean Owen |
| SPARK-15351 | RowEncoder should support array as the external type for ArrayType | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-15333 | Reorganize building-spark.md; rationalize vs wiki | Minor | Documentation | Sean Owen | Sean Owen |
| SPARK-15182 | Copy MLlib doc to ML: ml.feature | Minor | ML | yuhao yang | yuhao yang |
| SPARK-11735 | Add a check in the constructor of SqlContext to make sure the SparkContext is not stopped | Major | SQL | Yin Huai | Shixiong Zhu |
| SPARK-14978 | PySpark TrainValidationSplitModel should support validationMetrics | Major | PySpark | Kai Jiang | Takuya Kuwahara |
| SPARK-15322 | update deprecate accumulator usage into accumulatorV2 in mllib | Minor | ML, MLlib | Weichen Xu | Weichen Xu |
| SPARK-15346 | Reduce duplicate computation in picking initial points in LocalKMeans | Minor | . | Abraham Zhan | Abraham Zhan |
| SPARK-14939 | Add FoldablePropagation optimizer | Major | Optimizer | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-15362 | Make spark.ml KMeansModel load backwards compatible | Major | ML | Yanbo Liang | Yanbo Liang |
| SPARK-15292 | ML 2.0 QA: Scala APIs audit for classification | Major | ML | Yanbo Liang | Yanbo Liang |
| SPARK-15361 | ML 2.0 QA: Scala APIs audit for clustering | Minor | ML | Yanbo Liang | Yanbo Liang |
| SPARK-15414 | Make the mllib,ml linalg type conversion APIs public | Major | ML, MLlib | Joseph K. Bradley | Sandeep Singh |
| SPARK-15416 | Display a better message for not finding classes removed in Spark 2.0 | Major | SQL | Shixiong Zhu | Shixiong Zhu |
| SPARK-15296 | Refactor All Java Tests that use SparkSession | Minor | ML, MLlib, Tests | Sandeep Singh | Sandeep Singh |
| SPARK-11827 | Support java.math.BigInteger in Type-Inference utilities for POJOs | Minor | SQL | Abhilash Srimat Tirumala Pallerlamudi | kevin yu |
| SPARK-15335 | Implement TRUNCATE TABLE Command | Minor | SQL | Weizhong | Lianhui Wang |
| SPARK-15363 | Example code shouldn't use VectorImplicits._, asML/fromML | Major | Documentation, ML | Xiangrui Meng | Miao Wang |
| SPARK-15398 | Update the warning message to recommend ML usage | Minor | ML | zhengruifeng | zhengruifeng |
| SPARK-15339 | ML 2.0 QA: Scala APIs and code audit for regression | Major | ML | Yanbo Liang | Yanbo Liang |
| SPARK-15222 | SparkR ML examples update in 2.0 | Minor | ML, SparkR | Yanbo Liang | Yanbo Liang |
| SPARK-15438 | Improve the explain of whole-stage codegen | Major | SQL | Davies Liu | Davies Liu |
| SPARK-15205 | Codegen can compile the same source code more than twice | Major | SQL | Kousuke Saruta | Kousuke Saruta |
| SPARK-15280 | Extract ORC serialization logic from OrcOutputWriter for reusability | Minor | Input/Output | Ergin Seyfe | Ergin Seyfe |
| SPARK-15471 | ScalaReflection cleanup | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-15431 | Support LIST FILE(s)|JAR(s) command natively | Major | SQL | Xin Wu | Xin Wu |
| SPARK-15464 | Replace SQLContext and SparkContext with SparkSession using builder pattern in python testsuites | Minor | ML, MLlib, SQL, Tests | Weichen Xu | Weichen Xu |
| SPARK-15442 | PySpark QuantileDiscretizer missing "relativeError" param | Minor | ML, PySpark | Nick Pentreath | Nick Pentreath |
| SPARK-12949 | Support common expression elimination | Major | SQL | Davies Liu | Liang-Chi Hsieh |
| SPARK-15365 | Metastore relation should fallback to HDFS size if statistics are not available from table meta data. | Major | SQL | Parth Brahmbhatt | Parth Brahmbhatt |
| SPARK-15498 | fix slow tests | Major | Tests | Wenchen Fan | Wenchen Fan |
| SPARK-15412 | Improve linear & isotonic regression methods PyDocs | Trivial | Documentation, PySpark | holdenk | holdenk |
| SPARK-15537 | clean up the temp folders after finishing the tests | Major | SQL | Bo Meng | Bo Meng |
| SPARK-15542 | Make error message clear for script './R/install-dev.sh' when R is missing on Mac | Minor | SparkR | Xin Ren | Xin Ren |
| SPARK-15584 | Abstract duplicate code: "spark.sql.sources." properties | Minor | SQL | Andrew Or | Dongjoon Hyun |
| SPARK-15484 | Document Iteratively reweighted least squares (IRLS) in user guide | Minor | Documentation, ML | Yanbo Liang | Yanbo Liang |
| SPARK-15413 | Change toBreeze to asBreeze in Vector and Matrix |
Major | ML, MLlib | DB Tsai | DB Tsai |
| SPARK-15645 | Fix some typos of Streaming module | Trivial | Streaming | Xin Ren | Xin Ren |
| SPARK-15641 | Incorrect Completed for Incomplete applications in HistoryServer | Trivial | . | Jacek Laskowski | Cedar Pan |
| SPARK-15649 | Avoid serializing MetastoreRelation in HiveTableScanExec | Minor | . | Lianhui Wang | Lianhui Wang |
| SPARK-15670 | Add deprecate annotation for acumulator V1 interface in JavaSparkContext class | Minor | Java API, Spark Core | Weichen Xu | Weichen Xu |
| SPARK-14118 | Implement DDL/DML commands for Spark 2.0 | Blocker | SQL | Yin Huai | Yin Huai |
| SPARK-15139 | PySpark TreeEnsemble missing methods | Minor | ML, PySpark | holdenk | holdenk |
| SPARK-15734 | Avoids printing internal row in explain output | Minor | SQL | Sean Zhong | Xiang Zhong |
| SPARK-15718 | better error message for writing bucketing data | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-15733 | Makes the explain output less verbose by hiding some verbose output like None, null, empty List, and etc.. | Minor | SQL | Sean Zhong | Xiang Zhong |
| SPARK-15494 | encoder code cleanup | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-15737 | Fix Jetty server start warning | Minor | Spark Core | Bo Meng | Bo Meng |
| SPARK-15681 | Allow case-insensitiveness in sc.setLogLevel | Minor | Spark Core | Xin Wu | Xin Wu |
| SPARK-15168 | Add missing params to Python's MultilayerPerceptronClassifier | Trivial | ML, PySpark | holdenk | holdenk |
| SPARK-15756 | Support command 'create table stored as orcfile/parquetfile/avrofile' | Minor | . | marymwu | Lianhui Wang |
| SPARK-15707 | Make Code Neat - Use map instead of if check | Trivial | SQL | Weiqing Yang | Weiqing Yang |
| SPARK-15770 | annotation audit for Experimental and DeveloperApi | Trivial | ML | zhengruifeng | zhengruifeng |
| SPARK-14279 | Improve the spark build to pick the version information from the pom file and add git commit information | Minor | Build | Sanket Reddy | Dhruve Ashar |
| SPARK-15721 | Make DefaultParamsReadable,Writable public APIs | Major | ML | Joseph K. Bradley | Joseph K. Bradley |
| SPARK-15778 | Add 2.0.0-preview to dropdown / reorg description of previews at spark.apache.org/downloads.html | Minor | Documentation | Sean Owen | Sean Owen |
| SPARK-15792 | [SQL] Allows operator to change the verbosity in explain output. | Minor | . | Sean Zhong | Sean Zhong |
| SPARK-15684 | Not mask startsWith and endsWith in R | Major | . | Miao Wang | Miao Wang |
| SPARK-13590 | Document the behavior of spark.ml logistic regression and AFT survival regression when there are constant features | Minor | ML | Xiangrui Meng | Yanbo Liang |
| SPARK-15789 | Allow reserved keywords in most places | Major | SQL | Herman van Hovell | Herman van Hovell |
| SPARK-15793 | Word2vec in ML package should have maxSentenceLength method | Minor | ML | Xusen Yin | Xusen Yin |
| SPARK-15788 | PySpark IDFModel missing "idf" property | Trivial | ML, PySpark | Nick Pentreath | Jeff Zhang |
| SPARK-15827 | Publish Spark's forked sbt-pom-reader to Maven Central | Major | Build, Project Infra | Josh Rosen | Josh Rosen |
| SPARK-15696 | Improve crosstab to have a consistent column order |
Major | SQL | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-15823 | Add @property for 'accuracy' in MulticlassMetrics | Trivial | ML, PySpark | zhengruifeng | zhengruifeng |
| SPARK-15837 | PySpark ML Word2Vec should support maxSentenceLength | Minor | ML, PySpark | Yanbo Liang | Weichen Xu |
| SPARK-15753 | Move some Analyzer stuff to Analyzer from DataFrameWriter | Major | SQL | Liang-Chi Hsieh | Liang-Chi Hsieh |
| SPARK-15871 | Add assertNotPartitioned check in DataFrameWriter | Major | SQL | Liwei Lin | Liwei Lin |
| SPARK-15875 | Avoid using Seq.length == 0 and Seq.lenth > 0. Use Seq.isEmpty and Seq.nonEmpty instead. | Minor | . | Yang Wang | Yang Wang |
| SPARK-15738 | PySpark ml.feature RFormula missing string representation displaying formula | Major | ML, PySpark | Bryan Cutler | Bryan Cutler |
| SPARK-15759 | Fallback to non-codegen if fail to compile generated code | Major | SQL | Davies Liu | Davies Liu |
| SPARK-15585 | Don't use null in data source options to indicate default value | Critical | SQL | Reynold Xin | Takeshi Yamamuro |
| SPARK-15807 | Support varargs for dropDuplicates in Dataset/DataFrame | Major | SQL | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-15860 | Metrics for codegen size and perf | Major | SQL | Eric Liang | Eric Liang |
| SPARK-15806 | Deprecate SPARK_MASTER_IP in favor of SPARK_MASTER_HOST | Minor | Documentation | Bo Meng | Bo Meng |
| SPARK-15813 | Spark Dyn Allocation Cancel log message misleading | Trivial | . | Peter Ableda | Peter Ableda |
| SPARK-15676 | Disallow Column Names as Partition Columns For Hive Tables | Major | SQL | Xiao Li | Xiao Li |
| SPARK-15887 | Bring back the hive-site.xml support for Spark 2.0 | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-15364 | Implement Python picklers for ml.Vector and ml.Matrix under spark.ml.python | Major | ML, PySpark | Xiangrui Meng | Liang-Chi Hsieh |
| SPARK-15932 | document the contract of encoder serializer expressions | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-15821 | Should we use mvn -T for multithreaded Spark builds? | Minor | Build | Adam Roberts | Adam Roberts |
| SPARK-15426 | Spark 2.0 SQL API audit | Blocker | SQL | Reynold Xin | Reynold Xin |
| SPARK-15457 | Eliminate MLlib 2.0 build warnings from deprecations | Blocker | ML, MLlib | Joseph K. Bradley | Joseph K. Bradley |
| SPARK-13498 | JDBCRDD should update some input metrics | Minor | SQL | Wayne Song | Wayne Song |
| SPARK-12492 | SQL page of Spark-sql is always blank | Major | SQL, Web UI | meiyoula | KaiXinXIaoLei |
| SPARK-15862 | Better Error Message When Having Database Name in CACHE TABLE AS SELECT | Minor | SQL | Xiao Li | Xiao Li |
| SPARK-15983 | Remove FileFormat.prepareRead() | Major | SQL | Cheng Lian | Cheng Lian |
| SPARK-15796 | Reduce spark.memory.fraction default to avoid overrunning old gen in JVM default config | Blocker | . | Gabor Feher | Sean Owen |
| SPARK-15868 | Executors table in Executors tab should sort Executor IDs in numerical order (not alphabetical order) | Minor | Web UI | Jacek Laskowski | Alex Bozarth |
| SPARK-15749 | Make the error message more meaningful | Trivial | SQL | Huaxin Gao | Huaxin Gao |
| SPARK-15501 | ML 2.0 QA: Scala APIs audit for recommendation | Blocker | Documentation, ML | Nick Pentreath | Nick Pentreath |
| SPARK-16008 | ML Logistic Regression aggregator serializes unnecessary data | Major | ML | Seth Hendrickson | Seth Hendrickson |
| SPARK-15603 | Replace SQLContext with SparkSession in ML/MLLib | Major | ML, MLlib | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-15803 | Support with statement syntax for SparkSession | Minor | PySpark | Jeff Zhang | Jeff Zhang |
| SPARK-16023 | Move InMemoryRelation to its own file | Minor | SQL | Andrew Or | Andrew Or |
| SPARK-15973 | Fix GroupedData Documentation | Trivial | PySpark | Vladimir Feinberg | Josh Howes |
| SPARK-15942 | Unblock :reset command in REPL. |
Minor | . | Prashant Sharma | Prashant Sharma |
| SPARK-16059 | Add monotonically\_increasing\_id function in SparkR |
Minor | SparkR | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-16051 | Add read.orc/write.orc to SparkR |
Major | SparkR | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-16053 | Add spark\_partition\_id in SparkR |
Minor | SparkR | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-14995 | Add "since" tag in Roxygen documentation for SparkR API methods | Major | SparkR | Sun Rui | Dongjoon Hyun |
| SPARK-16061 | The property "spark.streaming.stateStore.maintenanceInterval" should be renamed to "spark.sql.streaming.stateStore.maintenanceInterval" | Minor | SQL | Kousuke Saruta | Kousuke Saruta |
| SPARK-15294 | Add pivot functionality to SparkR | Minor | SparkR | Mikołaj Hnatiuk | Dongjoon Hyun |
| SPARK-16045 | Spark 2.0 ML.feature: doc update for stopwords and binarizer | Minor | ML | yuhao yang | yuhao yang |
| SPARK-16084 | Minor javadoc issue with "Describe" table in the parser | Trivial | SQL | Bo Meng | Bo Meng |
| SPARK-15741 | PySpark Cleanup of _setDefault with seed=None | Minor | ML, PySpark | Bryan Cutler | Bryan Cutler |
| SPARK-16117 | Hide LibSVMFileFormat in public API docs | Major | Documentation, MLlib | Xiangrui Meng | Xiangrui Meng |
| SPARK-15644 | Replace SQLContext with SparkSession in MLlib | Major | MLlib, SQL | Xiao Li | Xiao Li |
| SPARK-15162 | Update PySpark LogisticRegression threshold PyDoc to be as complete as Scaladoc | Trivial | . | holdenk | holdenk |
| SPARK-16155 | Remove package grouping in genjavadoc | Major | Documentation | Xiangrui Meng | Xiangrui Meng |
| SPARK-16162 | Remove dead code: class OrcTableScan | Minor | SQL | Brian Cho | Brian Cho |
| SPARK-16154 | Update spark.ml and spark.mllib package docs | Major | Documentation, MLlib | Xiangrui Meng | Xiangrui Meng |
| SPARK-16130 | model loading backward compatibility for ml.classfication.LogisticRegression | Minor | ML | yuhao yang | yuhao yang |
| SPARK-13723 | YARN - Change behavior of --num-executors when spark.dynamicAllocation.enabled true | Minor | YARN | Thomas Graves | Ryan Blue |
| SPARK-16177 | model loading backward compatibility for ml.regression | Minor | ML | yuhao yang | yuhao yang |
| SPARK-16133 | model loading backward compatibility for ml.feature | Major | ML | yuhao yang | yuhao yang |
| SPARK-16135 | Remove hashCode and euqals in ArrayBasedMapData | Major | SQL | Takeshi Yamamuro | Takeshi Yamamuro |
| SPARK-16221 | Redirect Parquet JUL logger via SLF4J for WRITE operations | Minor | SQL | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-16259 | Cleanup options for DataFrame reader API in Python | Major | PySpark | Davies Liu | Davies Liu |
| SPARK-16175 | Handle None for all Python UDT | Major | PySpark, SQL | Davies Liu | Davies Liu |
| SPARK-16248 | Whitelist the list of Hive fallback functions | Major | . | Reynold Xin | Reynold Xin |
| SPARK-16245 | model loading backward compatibility for ml.feature.PCA | Minor | ML | Yanbo Liang | Yanbo Liang |
| SPARK-16238 | Metrics for generated method bytecode size | Minor | Spark Core | Eric Liang | Eric Liang |
| SPARK-12177 | Update KafkaDStreams to new Kafka 0.10 Consumer API | Major | Streaming | Nikita Tarasenko | Cody Koeninger |
| SPARK-16241 | model loading backward compatibility for ml NaiveBayes | Minor | ML | yuhao yang | Li Ping Zhang |
| SPARK-15643 | ML 2.0 QA: migration guide update | Blocker | Documentation, ML, MLlib | Yanbo Liang | Yanbo Liang |
| SPARK-15761 | pyspark shell should load if PYSPARK_DRIVER_PYTHON is ipython an Python3 | Minor | PySpark | Manoj Kumar | Manoj Kumar |
| SPARK-16012 | add gapplyCollect() for SparkDataFrame | Major | SparkR | Sun Rui | Narine Kokhlikyan |
| SPARK-16345 | Extract graphx programming guide example snippets from source files instead of hard code them | Major | Documentation, Examples, GraphX | Weichen Xu | Weichen Xu |
| SPARK-16249 | Change visibility of Object ml.clustering.LDA to public for loading | Minor | ML | yuhao yang | yuhao yang |
| SPARK-13569 | Kafka DStreams from wildcard topic filters | Major | Streaming | Miguel Miranda | Cody Koeninger |
| SPARK-16476 | Restructure MimaExcludes for easier union excludes | Major | Build | Reynold Xin | Reynold Xin |
| SPARK-16270 | Implement xpath user defined functions | Major | SQL | Peter Lee | Peter Lee |
| SPARK-16458 | SessionCatalog should support listColumns for temporary tables |
Minor | SQL | Dongjoon Hyun | Dongjoon Hyun |
| SPARK-16470 | ml.regression.LinearRegression training data do not check whether the result actually reach convergence | Trivial | ML, Optimizer | Weichen Xu | Weichen Xu |
| SPARK-16469 | Long running Driver task while multiplying big matrices | Minor | MLlib | Ohad Raviv | Ohad Raviv |
| SPARK-16500 | Add LBFG training not convergence warning for all ML algorithm | Trivial | ML, MLlib, Optimizer | Weichen Xu | Weichen Xu |
| SPARK-16582 | Explicitly define isNull = false for non-nullable expressions | Major | SQL | Sameer Agarwal | Sameer Agarwal |
| SPARK-16584 | Move regexp unit tests to RegexpExpressionsSuite | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-16588 | Deprecate monotonicallyIncreasingId in Scala | Trivial | SQL | Hyukjin Kwon | Reynold Xin |
| SPARK-16615 | Expose sqlContext in SparkSession | Major | . | Reynold Xin | Reynold Xin |
| SPARK-6744 | Add support for CROSS JOIN syntax | Minor | SQL | Santiago M. Mola | Herman van Hovell |
| JIRA | Summary | Priority | Component | Reporter | Contributor |
|---|---|---|---|---|---|
| SPARK-11511 | Creating an InputDStream but not using it throws NPE | Major | Streaming | Shixiong Zhu | Shixiong Zhu |
| SPARK-10116 | XORShiftRandom should generate uniform seeds | Minor | Spark Core | Imran Rashid | Imran Rashid |
| SPARK-11218 | ./sbin/start-slave.sh --help should print out the help message |
Minor | Deploy | Jacek Laskowski | Charles Yeh |
| SPARK-11500 | Not deterministic order of columns when using merging schemas. | Major | SQL | Hyukjin Kwon | Hyukjin Kwon |
| SPARK-11762 | TransportResponseHandler should consider open streams when counting outstanding requests | Minor | Spark Core | Marcelo Vanzin | Marcelo Vanzin |
| SPARK-11043 | Hive Thrift Server will log warn "Couldn't find log associated with operation handle" | Major | SQL | carlmartin | carlmartin |
| SPARK-11956 | Test failures potentially related to SPARK-11140 | Major | Spark Core | Marcelo Vanzin | Marcelo Vanzin |
| SPARK-11969 | SQL UI does not work with PySpark | Critical | SQL | Davies Liu | Davies Liu |
| SPARK-12003 | Expanded star should use field name as column name | Major | . | Davies Liu | Davies Liu |
| SPARK-11991 | spark_ec2.py does not perform sanity checks on hostnames | Major | EC2 | Jeremy Derr | Jeremy Derr |
| SPARK-12028 | [SQL] get_json_object is unable to return a correct result for null literals | Major | SQL | Xiao Li | Xiao Li |
| SPARK-12090 | Coalesce does not consider shuffle in PySpark | Major | . | Davies Liu | Davies Liu |
| SPARK-12001 | StreamingContext cannot be completely stopped if the stop() call is interrupted | Major | Streaming | Josh Rosen | Josh Rosen |
| SPARK-12082 | NettyBlockTransferSecuritySuite "security mismatch auth off on client" test is flaky | Major | Tests | Josh Rosen | Josh Rosen |
| SPARK-12109 | Expressions's simpleString should delegate to its toString | Major | SQL | Yin Huai | Yin Huai |
| SPARK-12088 | check connection.isClose before connection.getAutoCommit in JDBCRDD.close | Minor | SQL | Huaxin Gao | Huaxin Gao |
| SPARK-12107 | Update spark-ec2 versions | Minor | EC2 | Nicholas Chammas | Nicholas Chammas |
| SPARK-12019 | SparkR.init does not support character vector for sparkJars and sparkPackages | Minor | SparkR | liushiqi9 | Felix Cheung |
| SPARK-12104 | collect() does not handle multiple columns with same name | Critical | SparkR | Hossein Falaki | Sun Rui |
| SPARK-12112 | Upgrade to SBT 0.13.9 | Major | Build | Josh Rosen | Josh Rosen |
| SPARK-12142 | Can't request executor when container allocator is not ready | Major | Spark Core | meiyoula | meiyoula |
| SPARK-12084 | Fix codes that uses ByteBuffer.array incorrectly | Major | Spark Core | Shixiong Zhu | Shixiong Zhu |
| SPARK-11994 | Word2VecModel load and save cause SparkException when model is bigger than spark.kryoserializer.buffer.max | Major | MLlib | Antonio Murgia | Antonio Murgia |
| SPARK-11715 | R support corr for Column Aggregration | Minor | SparkR | Felix Cheung | Felix Cheung |
| SPARK-12048 | JDBCRDD calls close() twice - SQLite then throws an exception | Minor | SQL | R. H. | R. H. |
| SPARK-12132 | Cltr-C should clear current line in pyspark shell | Major | PySpark | Davies Liu | Davies Liu |
| SPARK-12222 | deserialize RoaringBitmap using Kryo serializer throw Buffer underflow exception | Major | Spark Core | Fei Wang | Fei Wang |
| SPARK-12031 | Integer overflow when do sampling. | Major | Spark Core | Genmao Yu | Genmao Yu |
| SPARK-10582 | using dynamic-executor-allocation, if AM failed. the new AM will be started. But the new AM does not allocate executors to dirver | Major | Spark Core | KaiXinXIaoLei | Saisai Shao |
| SPARK-12131 | Cannot create ExpressionEncoder for Array[T] where T is a nested class | Major | SQL | Cheng Lian | Cheng Lian |
| SPARK-12136 | rddToFileName does not properly handle prefix and suffix parameters | Minor | Streaming | Brian Webb | Bo Meng |
| SPARK-11832 | Spark shell does not work from sbt with scala 2.11 | Minor | Spark Shell | Jakob Odersky | Jakob Odersky |
| SPARK-11497 | PySpark RowMatrix Constructor Has Type Erasure Issue | Minor | MLlib, PySpark | Mike Dusenberry | Mike Dusenberry |
| SPARK-12298 | Infinite loop in DataFrame.sortWithinPartitions(String, String*) | Major | SQL | Ankur Dave | Ankur Dave |
| SPARK-12158 | [R] [SQL] Fix 'sample' functions that break R unit test cases | Critical | SparkR, SQL | Xiao Li | Xiao Li |
| SPARK-11193 | Spark 1.5+ Kinesis Streaming - ClassCastException when starting KinesisReceiver | Major | Streaming | Phil Kallos | Jean-Baptiste Onofré |
| SPARK-12275 | No plan for BroadcastHint in some condition | Major | SQL | yucai | yucai |
| SPARK-12271 | Improve error message for Dataset.as[] when the schema is incompatible. | Major | SQL | Nong Li | Apache Spark |
| SPARK-9026 | SimpleFutureAction.onComplete should not tie up a separate thread for each callback | Major | Spark Core | Josh Rosen | Richard W. Eggert II |
| SPARK-9886 | Validate usages of Runtime.getRuntime.addShutdownHook | Minor | Spark Core | Michel Lemay | Naveen Kumar Minchu |
| SPARK-12062 | Master rebuilding historical SparkUI should be asynchronous | Major | Deploy | Andrew Or | Bryan Cutler |
| SPARK-4514 | SparkContext localProperties does not inherit property updates across thread reuse | Critical | Spark Core | Erik Erlandson | Richard W. Eggert II |
| SPARK-12324 | The documentation sidebar does not collapse properly | Minor | Documentation | Timothy Hunter | Timothy Hunter |
| SPARK-12318 | Save mode in SparkR should be error by default | Minor | SparkR | Jeff Zhang | Jeff Zhang |
| SPARK-12380 | MLLib should use existing SQLContext instead create new one | Major | MLlib, PySpark | Davies Liu | Davies Liu |
| SPARK-12365 | Use ShutdownHookManager where Runtime.getRuntime.addShutdownHook() is called | Minor | Spark Core | Ted Yu | Ted Yu |
| SPARK-12186 | stage web URI will redirect to the wrong location if it is the first URI from the application to be requested from the history server | Minor | Web UI | Rohit Agarwal | Rohit Agarwal |
| SPARK-12395 | Result of DataFrame.join(usingColumns) could be wrong for outer join | Blocker | SQL | Davies Liu | Davies Liu |
| SPARK-12390 | Clean up unused serializer parameter in BlockManager | Major | Block Manager, Spark Core | Andrew Or | Andrew Or |
| SPARK-12350 | VectorAssembler#transform() initially throws an exception | Major | Spark Core, Spark Shell | Jakob Odersky | Marcelo Vanzin |
| SPARK-11619 | cannot use UDTF in DataFrame.selectExpr | Minor | SQL | Wenchen Fan | Dilip Biswal |
| SPARK-12218 | Invalid splitting of nested AND expressions in Data Source filter API | Blocker | SQL | Irakli Machabeli | Yin Huai |
| SPARK-12091 | [PySpark] Removal of the JAVA-specific deserialized storage levels | Minor | PySpark | Xiao Li | Xiao Li |
| SPARK-12442 | Build testing failed | Blocker | Build | Xiao Li | Reynold Xin |
| SPARK-12466 | Harmless Master NPE in tests | Major | Deploy, Tests | Andrew Or | Andrew Or |
| SPARK-12339 | NullPointerException on stage kill from web UI | Major | Web UI | Jacek Laskowski | Alex Bozarth |
| SPARK-12102 | Cast a non-nullable struct field to a nullable field during analysis | Major | SQL | Yin Huai | Dilip Biswal |
| SPARK-12477 | [SQL] Tungsten projection fails for null values in array fields | Major | SQL | Pierre Borckmans | Pierre Borckmans |
| SPARK-12499 | make_distribution should not override MAVEN_OPTS | Minor | Build | Adrian Bridgett | Adrian Bridgett |
| SPARK-12502 | Script /dev/run-tests fails when IBM Java is used | Minor | Tests | Kazuaki Ishizaki | Kazuaki Ishizaki |
| SPARK-12311 | [CORE] Restore previous value of "os.arch" property in test suites after forcing to set specific value to "os.arch" property | Minor | Spark Core | Kazuaki Ishizaki | Kazuaki Ishizaki |
| SPARK-12010 | Spark JDBC requires support for column-name-free INSERT syntax | Major | SQL | Christian Kurz | Christian Kurz |
| SPARK-12396 | Once driver client registered successfully,it still retry to connected to master. | Minor | Spark Core | echo | echo |
| SPARK-12520 | Python API dataframe join returns wrong results on outer join | Major | PySpark, SQL | Aravind B | Xiao Li |
| SPARK-12353 | wrong output for countByValue and countByValueAndWindow | Major | Documentation, Input/Output, PySpark, Streaming | Bo Jin | Saisai Shao |
| SPARK-12508 | Fix multiple bugs in pr_public_classes.sh script | Trivial | Project Infra | Josh Rosen | Apache Spark |
| SPARK-12231 | Failed to generate predicate Error when using dropna | Major | PySpark, SQL | yahsuan, chang | kevin yu |
| SPARK-12517 | No default RDD name for ones created by sc.textFile | Minor | Spark Core | yaron weinsberg | yaron weinsberg |
| SPARK-12424 | The implementation of ParamMap#filter is wrong. | Major | ML | Kousuke Saruta | Kousuke Saruta |
| SPARK-12441 | Fixing missingInput in all Logical/Physical operators | Major | SQL | Xiao Li | Xiao Li |
| SPARK-12525 | Fix compiler warnings in Kinesis ASL module due to @transient annotations | Major | Streaming | Josh Rosen | Josh Rosen |
| SPARK-12489 | Fix minor issues found by Findbugs | Minor | MLlib, Spark Core, SQL | Shixiong Zhu | Shixiong Zhu |
| SPARK-12522 | Add the missing the document string for the SQL configuration | Minor | SQL | Xiao Li | Xiao Li |
| SPARK-11394 | PostgreDialect cannot handle BYTE types | Major | SQL | Takeshi Yamamuro | Takeshi Yamamuro |
| SPARK-12526 | ifelse , when , otherwise unable to take Column as value |
Major | SparkR | Sen Fang | Sen Fang |
| SPARK-12530 | Build break at Spark-Master-Maven-Snapshots from #1293 | Major | Build | Kazuaki Ishizaki | Kazuaki Ishizaki |
| SPARK-12300 | Fix schema inferance on local collections | Minor | PySpark, SQL | holdenk | holdenk |
| SPARK-12399 | Display correct error message when accessing REST API with an unknown app Id | Minor | Web UI | Carson Wang | Carson Wang |
| SPARK-12039 | HiveSparkSubmitSuite's SPARK-9757 Persist Parquet relation with decimal column is very flaky | Major | SQL, Tests | Yin Huai | Yin Huai |
| SPARK-12327 | lint-r checks fail with commented code | Major | SparkR | Shivaram Venkataraman | Felix Cheung |
| SPARK-12533 | hiveContext.table() throws the wrong exception | Major | SQL | Michael Armbrust | Thomas Sebastian |
| SPARK-12611 | test_infer_schema_to_local depended on old handling of missing value in row | Trivial | SQL, Tests | holdenk | holdenk |
| SPARK-12562 | DataFrame.write.format("text") requires the column name to be called value | Major | SQL | Michael Armbrust | Xiu (Joe) Guo |
| SPARK-12612 | Add missing Hadoop profiles to dev/run-tests-*.py scripts | Major | Build, Project Infra | Josh Rosen | Josh Rosen |
| SPARK-12513 | SocketReceiver hang in Netcat example | Minor | Streaming | Shawn Guo | Shawn Guo |
| SPARK-12579 | User-specified JDBC driver should always take precedence | Major | SQL | Josh Rosen | Josh Rosen |
| SPARK-12470 | Incorrect calculation of row size in o.a.s.sql.catalyst.expressions.codegen.GenerateUnsafeRowJoiner | Minor | SQL | Pete Robbins | Pete Robbins |
| SPARK-12512 | WithColumn does not work on multiple column with special character | Major | SQL | JO EE | Xiu (Joe) Guo |
| SPARK-12421 | Fix copy() method of GenericRow | Minor | SQL | doepfner | Herman van Hovell |
| SPARK-12589 | result row size is wrong in UnsafeRowParquetRecordReader | Major | SQL | Wenchen Fan | Nong Li |
| SPARK-12509 | Fix error messages for DataFrame correlation and covariance | Minor | Documentation, SQL | Narine Kokhlikyan | Narine Kokhlikyan |
| SPARK-12625 | SparkR is using deprecated APIs | Blocker | SparkR, SQL | Reynold Xin | Felix Cheung |
| SPARK-12439 | Fix toCatalystArray and MapObjects | Major | SQL | Liang-Chi Hsieh | Liang-Chi Hsieh |
| SPARK-12617 | socket descriptor leak killing streaming app | Critical | PySpark, Streaming | Antony Mayi | Shixiong Zhu |
| SPARK-12511 | streaming driver with checkpointing unable to finalize leading to OOM | Critical | PySpark, Streaming | Antony Mayi | Shixiong Zhu |
| SPARK-12453 | Spark Streaming Kinesis Example broken due to wrong AWS Java SDK version | Critical | Streaming | Martin Schade | Brian London |
| SPARK-12504 | JDBC data source credentials are not masked in the data frame explain output. | Major | SQL | Suresh Thalamati | Suresh Thalamati |
| SPARK-12659 | NPE when spill in CartisianProduct | Major | SQL | Davies Liu | Davies Liu |
| SPARK-12340 | overstep the bounds of Int in SparkPlan.executeTake | Major | SQL | QiangCai | QiangCai |
| SPARK-12673 | Prepending base URI of job description is missing | Major | Web UI | Saisai Shao | Saisai Shao |
| SPARK-12678 | MapPartitionsRDD should clear reference to prev RDD | Minor | Spark Core | Guillaume Poulin | Guillaume Poulin |
| SPARK-12006 | GaussianMixture.train crashes if an initial model is not None | Major | MLlib, PySpark | Maciej Szymkiewicz | Maciej Szymkiewicz |
| SPARK-12662 | Add a local sort operator to DataFrame used by randomSplit | Major | Documentation, SQL | Yin Huai | Sameer Agarwal |
| SPARK-12598 | Bug in setMinPartitions function of StreamFileInputFormat | Minor | Spark Core | Darek Blasiak | Darek Blasiak |
| SPARK-12591 | NullPointerException using checkpointed mapWithState with KryoSerializer | Major | Streaming | Jan Uyttenhove | Shixiong Zhu |
| SPARK-12654 | sc.wholeTextFiles with spark.hadoop.cloneConf=true fails on secure Hadoop | Major | Spark Core | Thomas Graves | Thomas Graves |
| SPARK-12736 | Standalone Master cannot be started due to NoClassDefFoundError: org/spark-project/guava/collect/Maps | Major | Deploy, Spark Core | Jacek Laskowski | Jacek Laskowski |
| SPARK-12734 | Fix Netty exclusions and use Maven Enforcer to prevent bug from being reintroduced | Major | Build, Project Infra | Josh Rosen | Josh Rosen |
| SPARK-12744 | Inconsistent behavior parsing JSON with unix timestamp values | Minor | SQL | Anatoliy Plastinin | Anatoliy Plastinin |
| SPARK-12742 | org.apache.spark.sql.hive.LogicalPlanToSQLSuite failure due to Table already exists | Major | SQL | Fei Wang | Fei Wang |
| SPARK-12638 | Parameter explaination not very accurate for rdd function "aggregate" | Trivial | Documentation, Spark Core | Tommy Yu | Tommy Yu |
| SPARK-12582 | IndexShuffleBlockResolverSuite fails in windows | Major | Tests, Windows | yucai | yucai |
| SPARK-7615 | MLLIB Word2Vec wordVectors divided by Euclidean Norm equals to zero | Minor | MLlib | Eric Li | Sean Owen |
| SPARK-12652 | Upgrade py4j to the incoming version 0.9.1 | Major | PySpark | Shixiong Zhu | Shixiong Zhu |
| SPARK-12558 | AnalysisException when multiple functions applied in GROUP BY clause | Major | SQL | Maciej Bryński | Dilip Biswal |
| SPARK-12805 | Outdated details in doc related to Mesos run modes | Minor | Documentation | Luc Bourlier | Luc Bourlier |
| SPARK-12685 | word2vec trainWordsCount gets overflow | Minor | MLlib | yuhao yang | yuhao yang |
| SPARK-12268 | pyspark shell uses execfile which breaks python3 compatibility | Major | PySpark | Erik Selin | Erik Selin |
| SPARK-12690 | NullPointerException in UnsafeInMemorySorter.free() | Minor | Spark Core | Carson Wang | Carson Wang |
| SPARK-12478 | Dataset fields of product types can't be null | Major | SQL | Cheng Lian | Apache Spark |
| SPARK-12026 | ChiSqTest gets slower and slower over time when number of features is large | Major | MLlib | Hunter Kelly | yuhao yang |
| SPARK-9844 | File appender race condition during SparkWorker shutdown | Major | Spark Core | Alex Liu | Bryan Cutler |
| SPARK-12784 | Spark UI IndexOutOfBoundsException with dynamic allocation | Major | Web UI, YARN | Thomas Graves | Shixiong Zhu |
| SPARK-12821 | Style checker should run when some configuration files for style are modified but any source files are not. | Minor | Project Infra | Kousuke Saruta | Kousuke Saruta |
| SPARK-12708 | Sorting task error in Stages Page when yarn mode | Minor | Web UI | Koyo Yoshida | Koyo Yoshida |
| SPARK-12655 | GraphX does not unpersist RDDs | Minor | GraphX | Alexander Pivovarov | Jason C Lee |
| SPARK-12842 | Add Hadoop 2.7 build profile | Major | Build | Josh Rosen | Josh Rosen |
| SPARK-12862 | Jenkins does not run R tests | Major | SparkR | Felix Cheung | Felix Cheung |
| SPARK-12346 | GLM summary crashes with NoSuchElementException if attributes are missing names | Major | SparkR | Eric Liang | Eric Liang |
| SPARK-12841 | UnresolvedException with cast | Blocker | SQL | Michael Armbrust | Wenchen Fan |
| SPARK-12870 | better format bucket id in file name | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-12804 | ml.classification.LogisticRegression fails when FitIntercept with same-label dataset | Major | ML | Feynman Liang | Feynman Liang |
| SPARK-12867 | Nullability of Intersect can be stricter | Minor | SQL | Cheng Lian | Xiao Li |
| SPARK-12816 | Schema generation for type aliases does not work | Major | Spark Core | Jakob Odersky | Jakob Odersky |
| SPARK-12168 | Need test for conflicted function in R | Minor | SparkR | Felix Cheung | Felix Cheung |
| SPARK-12232 | Create new R API for read.table to avoid conflict | Minor | SparkR | Felix Cheung | Felix Cheung |
| SPARK-12881 | Enable sub-expression Elimination in generated mutable projection | Major | SQL | Davies Liu | Davies Liu |
| SPARK-12230 | WeightedLeastSquares.fit() should handle division by zero properly if standard deviation of target variable is zero. | Trivial | ML | Imran Younus | Imran Younus |
| SPARK-12960 | Some examples are missing support for python2 | Minor | PySpark | Mark Grover | Mark Grover |
| SPARK-12959 | Writing Bucketed Data with Disabled Bucketing in SQLConf | Major | SQL | Xiao Li | Xiao Li |
| SPARK-12747 | Postgres JDBC ArrayType(DoubleType) 'Unable to find server array type' | Major | SQL | Brandon Bradley | Liang-Chi Hsieh |
| SPARK-12629 | SparkR: DataFrame's saveAsTable method has issues with the signature and HiveContext | Major | SparkR | Narine Kokhlikyan | Narine Kokhlikyan |
| SPARK-12859 | Names of input streams with receivers don't fit in Streaming page | Trivial | Streaming, Web UI | Jacek Laskowski | Alex Bozarth |
| SPARK-12760 | inaccurate description for difference between local vs cluster mode in closure handling | Minor | Documentation | Mortada Mehyar | Mortada Mehyar |
| SPARK-11137 | Make StreamingContext.stop() exception-safe | Minor | Streaming | Felix Cheung | Jayadevan M |
| SPARK-12904 | Strength reduction for integer/decimal comparisons | Major | Optimizer, SQL | Reynold Xin | Reynold Xin |
| SPARK-12971 | Address test isolation problems which broke Hive tests on Hadoop 2.3 SBT build | Major | SQL, Tests | Josh Rosen | Josh Rosen |
| SPARK-12624 | When schema is specified, we should give better error message if actual row length doesn't match | Blocker | PySpark, SQL | Reynold Xin | Cheng Lian |
| SPARK-12755 | Spark may attempt to rebuild application UI before finishing writing the event logs in possible race condition | Minor | Spark Core | Michael Allman | Michael Allman |
| SPARK-12961 | Work around memory leak in Snappy library | Major | Spark Core | Josh Rosen | Liang-Chi Hsieh |
| SPARK-12682 | Hive will fail if the schema of a parquet table has a very wide schema | Major | SQL | Yin Huai | Sameer Agarwal |
| SPARK-12952 | EMLDAOptimizer initialize should return EMLDAOptimizer other than its parent class | Trivial | MLlib | Xusen Yin | Xusen Yin |
| SPARK-12614 | Don't throw non fatal exception from RpcEndpointRef.send/ask | Major | Spark Core | Shixiong Zhu | Shixiong Zhu |
| SPARK-7780 | The intercept in LogisticRegressionWithLBFGS should not be regularized | Major | MLlib | DB Tsai | holdenk |
| SPARK-12780 | Inconsistency returning value of ML python models' properties | Minor | ML, PySpark | Xusen Yin | Xusen Yin |
| SPARK-13023 | Check for presence of 'root' module after computing test_modules, not changed_modules | Major | Project Infra | Josh Rosen | Josh Rosen |
| SPARK-10847 | Pyspark - DataFrame - Optional Metadata with None triggers cryptic failure |
Minor | PySpark, SQL | Shea Parkes | Jason C Lee |
| SPARK-13021 | Fail fast when custom RDD's violate RDD.partition's API contract | Major | Spark Core | Josh Rosen | Josh Rosen |
| SPARK-13067 | DataFrameSuite.simple explode fail locally | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-13050 | Scalatest tags fail builds with the addition of the sketch module | Major | Build | Alex Bozarth | Alex Bozarth |
| SPARK-10873 | Change history to use datatables to support sorting columns and searching | Major | Web UI | Thomas Graves | Zhuo Liu |
| SPARK-13055 | SQLHistoryListener throws ClassCastException | Major | Spark Core, SQL | Andrew Or | Andrew Or |
| SPARK-13082 | sqlCtx.real.json() doesn't work with PythonRDD | Major | PySpark | Gaëtan Lehmann | Shixiong Zhu |
| SPARK-13096 | Make AccumulatorSuite#verifyPeakExecutionMemorySet less flaky | Major | Tests | Andrew Or | Andrew Or |
| SPARK-13088 | DAG viz does not work with latest version of chrome | Blocker | Web UI | Andrew Or | Andrew Or |
| SPARK-13049 | Cannot use FIRST/LAST ignoreNulls option in 1.6 and below | Minor | SQL | Herman van Hovell | Herman van Hovell |
| SPARK-6847 | Stack overflow on updateStateByKey which followed by a dstream with checkpoint set | Critical | Streaming | Jack Hu | Shixiong Zhu |
| SPARK-12989 | Bad interaction between StarExpansion and ExtractWindowExpressions | Major | SQL | Michael Armbrust | Xiao Li |
| SPARK-10777 | order by fails when column is aliased and projection includes windowed aggregate | Major | SQL | N Campbell | Xiao Li |
| SPARK-10647 | Mesos HA mode misuses spark.deploy.zookeeper.dir property; configs should be documented | Minor | Mesos | Alan Braithwaite | Timothy Chen |
| SPARK-12265 | Spark calls System.exit inside driver instead of throwing exception | Major | Mesos | Iulian Dragos | Nilanjan Raychaudhuri |
| SPARK-12979 | Paths are resolved relative to the local file system | Major | Mesos | Iulian Dragos | Iulian Dragos |
| SPARK-13087 | Grouping by a complex expression may lead to incorrect AttributeReferences in aggregations | Critical | SQL | Mark Hamstra | Michael Armbrust |
| SPARK-12711 | ML StopWordsRemover does not protect itself from column name duplication | Trivial | ML, MLlib | Grzegorz Chilkiewicz | Grzegorz Chilkiewicz |
| SPARK-13121 | java mapWithState mishandles scala Option | Critical | Java API, Streaming | Gabriele Nizzoli | Gabriele Nizzoli |
| SPARK-13122 | Race condition in MemoryStore.unrollSafely() causes memory leak | Major | Spark Core, Streaming | Adam Budde | Adam Budde |
| SPARK-12732 | Fix LinearRegression.train for the case when label is constant and fitIntercept=false | Minor | MLlib | Imran Younus | Imran Younus |
| SPARK-13157 | ADD JAR command cannot handle path with @ character | Blocker | SQL | Cheng Lian | Herman van Hovell |
| SPARK-12739 | Details of batch in Streaming tab uses two Duration columns | Minor | Streaming, Web UI | Jacek Laskowski | Mario Briggs |
| SPARK-12330 | Mesos coarse executor does not cleanup blockmgr properly on termination if data is stored on disk | Major | Block Manager, Mesos | Charles Allen | Apache Spark |
| SPARK-13162 | Standalone mode does not respect spark.dynamicAllocation.initialExecutors |
Major | Deploy, Spark Core | Andrew Or | Andrew Or |
| SPARK-13053 | Rectify ignored tests in InternalAccumulatorSuite | Major | Spark Core, Tests | Andrew Or | Andrew Or |
| SPARK-13052 | waitingApps metric doesn't show the number of apps currently in the WAITING state | Minor | Web UI | Raafat Akkad | Raafat Akkad |
| SPARK-13187 | Add boolean/long/double type option functions in DataFrameReader/Writer | Major | SQL | Reynold Xin | Reynold Xin |
| SPARK-12939 | migrate encoder resolution to Analyzer | Major | SQL | Wenchen Fan | Wenchen Fan |
| SPARK-13002 | Mesos scheduler backend does not follow the property spark.dynamicAllocation.initialExecutors | Major | Mesos | Luc Bourlier | Luc Bourlier |
| SPARK-12986 | Fix pydoc warnings in mllib/regression.py | Minor | MLlib, PySpark | Xiangrui Meng | Nam Pham |
| SPARK-13101 | Dataset complex types mapping to DataFrame (element nullability) mismatch | Blocker | SQL | Deenar Toraskar | Wenchen Fan |
| SPARK-13210 | NPE in Sort | Critical | SQL | Davies Liu | Davies Liu |
| SPARK-13095 | improve performance of hash join with dimension table | Major | SQL | Davies Liu | Davies Liu |
| SPARK-12807 | Spark External Shuffle not working in Hadoop clusters with Jackson 2.2.3 | Critical | Shuffle, YARN | Steve Loughran | Steve Loughran |
| SPARK-13245 | ShuffleBlockFetcherIterator should not use shuffleMetrics in multiple threads | Major | Spark Core | Shixiong Zhu | Shixiong Zhu |
| SPARK-10524 | Decision tree binary classification with ordered categorical features: incorrect centroid | Major | ML, MLlib | Joseph K. Bradley | Liang-Chi Hsieh |
| SPARK-11518 | The script spark-submit.cmd can not handle spark directory with space. | Minor | Deploy, Windows | Cele Liu | Jon Maurer |
| SPARK-13254 | Fix planning of TakeOrderedAndProject operator | Major | SQL | Josh Rosen | Josh Rosen |
| SPARK-13163 | Column width on new History Server DataTables not getting set correctly | Minor | Web UI | Alex Bozarth | Alex Bozarth |
| SPARK-13126 | History Server page always has horizontal scrollbar | Minor | Web UI | Alex Bozarth | Zhuo Liu |
| SPARK-13056 | Map column would throw NPE if value is null | Major | SQL | Adrian Wang | Adrian Wang |
| SPARK-13276 | Parse Table Identifiers/Expression skips bad characters at the end of the passed string | Minor | SQL | Herman van Hovell | Herman van Hovell |
| SPARK-13270 | Improve readability of whole stage codegen by skipping empty lines and outputting the pipeline plan | Major | . | Nong Li | Nong Li |
| SPARK-13124 | Adding JQuery DataTables messed up the Web UI css and js | Major | Web UI | Alex Bozarth | Alex Bozarth |
| SPARK-13277 | ANTLR ignores other rule using the USING keyword | Minor | SQL | Herman van Hovell | Liang-Chi Hsieh |
| SPARK-13265 | Refactoring of basic ML import/export for other file system besides HDFS | Major | ML | Yu Ishikawa | Yu Ishikawa |
| SPARK-13047 | Pyspark Params.hasParam should not throw an error | Minor | ML, PySpark | Seth Hendrickson | Seth Hendrickson |
| SPARK-12765 | CountVectorizerModel.transform lost the transformSchema | Major | ML | sloth | sloth |
| SPARK-12746 | ArrayType(_, true) should also accept ArrayType(_, false) | Major | ML, SQL | Earthson Lu | Earthson Lu |
| SPARK-13153 | PySpark ML persistence failed when handle no default value parameter | Minor | ML, PySpark | Tommy Yu | Tommy Yu |
| SPARK-13260 | count(*) does not work with CSV data source | Major | SQL | Hossein Falaki | Hyukjin Kwon |