Apache Solr Release Notes
Introduction
Apache Solr is an open source enterprise search server based on the Apache Lucene Java
search library, with XML/HTTP and JSON APIs, hit highlighting, faceted search,
caching, replication, and a web administration interface.
See http://lucene.apache.org/solr for more information.
- Bug Fixes (8)
- SOLR-8460: /analysis/field could throw exceptions for custom attributes.
(David Smiley, Uwe Schindler)
- SOLR-8373: KerberosPlugin: Using multiple nodes on same machine leads clients to
fetch TGT for every request
(Ishan Chattopadhyaya via noble)
- SOLR-8059: &debug=results for distributed search when distrib.singlePass (sometimes activated
automatically) could result in an NPE.
(David Smiley, Markus Jelsma)
- SOLR-8422: When authentication enabled, requests fail if sent to a node that doesn't host
the collection
(noble)
- SOLR-7462: AIOOBE in RecordingJSONParser
(Scott Dawson, noble)
- SOLR-8496: SolrIndexSearcher.getDocSet(List<Query>) incorrectly included deleted documents
when all of the queries were uncached (or there was no filter cache). This caused
multi-select faceting (including the JSON Facet API) to include deleted doc counts
when the remaining non-excluded filters were all uncached. This bug was first introduced in 5.3.0
(Andreas Müller, Vasiliy Bout, Erick Erickson, Shawn Heisey, Hossman, yonik)
- SOLR-8418: Adapt to changes in LUCENE-6590 for use of boosts with MLTHandler and
Simple/CloudMLTQParser
(Jens Wille, Ramkumar Aiyengar)
- SOLR-8561: Doing a rolling upgrade to 5.4.0 fails because the new nodes cannot find the
shard leader properties where they expect. This bug was first introduced in 5.4.0 and a
fallback was added so that the leader properties are searched in both the old location
and the new one.
(Shai Erera)
- New Features (1)
- SOLR-8470: Make TTL of PKIAuthenticationPlugin's tokens configurable through a system property
(pkiauth.ttl)
(noble)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release
- Versions of Major Components (6)
- Apache Tika 1.7
- Carrot2 3.10.4
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.6
- Jetty 9.2.13.v20150730
- Upgrading from Solr 5.3 (4)
- DefaultSimilarityFactory has been renamed to ClassicSimilarityFactory to match the underlying rename of
DefaultSimilarity to ClassicSimilarity and the (eventual) move away from using it as a default.
If you currently have DefaultSimilarityFactory explicitly referenced in your schema.xml, you will now get
a warning urging you to edit your config to use the functionally identical ClassicSimilarityFactory.
DefaultSimilarityFactory will be removed completely in Solr 6. See SOLR-8239 for more details.
- SOLR-7859: The following APIs are now deprecated:
-
SolrCore.getStartTime: Use SolrCore.getStartTimeStamp instead.
-
SolrIndexSearcher.getOpenTime: Use SolrIndexSearcher.getOpenTimeStamp instead.
- SOLR-8307: EmptyEntityResolver was moved from core to solrj, and moved from the org.apache.solr.util
package to org.apache.solr.common. If you are using this class, you will need to adjust the import package.
- Logger declarations in most source files have changed to code that
no longer needs to explicitly state the class name. This fixes situations
where a logger for a different class was incorrectly used. See SOLR-8324
and its sub-issues for details.
Detailed Change List
- New Features (21)
- SOLR-5756: A utility Collection API to move a collection from shared clusterstate.json (stateFormat=1,
default until 4.x) to the per-collection state.json stored in ZooKeeper (stateFormat=2,
default since 5.0) seamlessly without any application down-time.
Example:
http://localhost:8983/solr/admin/collections?action=MIGRATESTATEFORMAT&collection=<collection_name>
(Noble Paul, Scott Blum, shalin)
- SOLR-7219: filterCache access added to the solr query syntax.
Example: description:HDTV OR filter(+promotion:tv +promotion_date:[NOW/DAY TO NOW/DAY+7DAY])
(yonik)
- SOLR-7775: Allow fromIndex parameter to ScoreJoinQParserPlugin {!join score=.. fromIndex=..}..
to refer to a single-sharded collection that has a replica on all nodes where there is a
replica in the to index
(Andrei Beliakov via Mikhail Khludnev)
- SOLR-7961: Print Solr's version with command bin/solr version
(janhoy)
- SOLR-7789: Introduce a ConfigSet management API
(Gregory Chanan)
- SOLR-4316: Add a collections dropdown to angular admin UI
(Upayavira, Shalin Shekhar Mangar)
- SOLR-7915: Provide pluggable context tool support for VelocityResponseWriter
(Erik Hatcher)
- LUCENE-6795: SystemInfoHandler was improved to also show detailed operating
system statistics on IBM J9 virtual machines. It also no longer fails on Java 9
with Jigsaw module system.
(Uwe Schindler)
- SOLR-8053: Basic auth support in SolrJ
(noble)
- SOLR-7995: Add a LIST command to ConfigSets API
(Gregory Chanan)
- SOLR-4388: In Angular UI, add a Collections UI when in cloud mode
(Upayavira)
- SOLR-7858, SOLR-8199: Add links between original and new Admin UIs
(Upayavira)
- SOLR-7888: Analyzing suggesters can now filter suggestions by a context field
(Arcadius Ahouansou, janhoy)
- SOLR-8217: JSON Facet API: add "method" param to terms/field facets to give an execution
hint for what method should be used to facet.
(yonik)
- SOLR-8113: CloneFieldUpdateProcessorFactory now supports choosing a "dest" field name based on a regex
pattern and replacement init options.
(Gus Heck, hossman)
- SOLR-8139: Create/delete fields/dynamic fields/copy fields via schema tab on Angular UI
- SOLR-8166: Introduce possibility to configure ParseContext in
ExtractingRequestHandler/ExtractingDocumentLoader
(Andriy Binetsky
via Uwe Schindler)
- SOLR-7569: A collection API to force elect a leader, called FORCELEADER, when all replicas in a shard are down
(Ishan Chattopadhyaya, Mark Miller, shalin, noble)
- SOLR-6168: Add a 'sort' local param to the collapse QParser to support using complex sort options
to select the representitive doc for each collapsed group.
(Umesh Prasad, hossman)
- SOLR-8329: SchemaSimilarityFactory now supports a 'defaultSimFromFieldType' init option for using
a fieldType name to identify which Similarity to use as a default.
(hossman)
- SOLR-7912: Add boost support, and also exclude the queried document in MoreLikeThis QParser
(Jens Wille via Anshum Gupta)
- Bug Fixes (59)
- SOLR-7859: Fix usage of currentTimeMillis instead of nanoTime in multiple places,
whitelist valid uses of currentTimeMillis
(Ramkumar Aiyengar)
- SOLR-7836: Possible deadlock when closing refcounted index writers.
(Jessica Cheng Mallet, Erick Erickson, Mark Miller, yonik)
- SOLR-7869: Overseer does not handle BadVersionException correctly and, in some cases,
can go into an infinite loop if cluster state in ZooKeeper is modified externally.
(Scott Blum, shalin)
- SOLR-7920: Resolve XSS issue in Admin UI Schema Browser
(David Chiu via Upayavira)
- SOLR-7935: Fix very rare race condition that can cause an update to fail
via NullPointerException during a core reload.
(yonik)
- SOLR-7941: multivalued params are concatenated when using config API
(noble)
- SOLR-7956: There are interrupts on shutdown in places that can cause ChannelAlreadyClosed
exceptions which prevents proper closing of transaction logs, interfere with the IndexWriter,
the hdfs client and other things.
(Mark Miller, Scott Blum)
- SOLR-7954: Fixed an integer overflow bug in the HyperLogLog code used by the 'cardinality' option
of stats.field to prevent ArrayIndexOutOfBoundsException in a distributed search when a large precision
is selected and a large number of values exist in each shard
(hossman)
- SOLR-7844: Zookeeper session expiry during shard leader election can cause multiple leaders.
(Mike Roberts, Mark Miller, Jessica Cheng)
- SOLR-7984: wrong and misleading error message 'no default request handler is registered'
(noble, hossman)
- SOLR-8001: Fixed bugs in field(foo,min) and field(foo,max) when some docs have no values
(David Smiley, hossman)
- SOLR-7819: ZK connection loss or session timeout do not stall indexing threads anymore. All activity
related to leader initiated recovery is performed by a dedicated LIR thread in the background.
(Ramkumar Aiyengar, shalin)
- SOLR-7746: Ping requests stopped working with distrib=true in Solr 5.2.1.
(Alexey Serba, Michael Sun via Gregory Chanan)
- SOLR-6547: ClassCastException in SolrResponseBase.getQTime on update response from CloudSolrClient
when parallelUpdates is enabled (default) and multiple docs are sent as a single update.
(kevin, hossman, shalin)
- SOLR-8058: Fix the exclusion filter so that collections that start with js, css, img, tpl
can be accessed.
(Upayavira, Steve Rowe, Anshum Gupta)
- SOLR-8069: Ensure that only the valid ZooKeeper registered leader can put a replica into Leader
Initiated Recovery.
(Mark Miller, Jessica Cheng, Anshum Gupta)
- SOLR-8077: Replication can still cause index corruption.
(Mark Miller)
- SOLR-8104: Config API does not work for spellchecker
(noble)
- SOLR-8095: Allow disabling HDFS Locality Metrics and disable by default as it may have performance
implications on rapidly changing indexes.
(Mike Drob via Mark Miller)
- SOLR-8085: Fix a variety of issues that can result in replicas getting out of sync.
(yonik, Mark Miller)
- SOLR-8094: HdfsUpdateLog should not replay buffered documents as a replacement to dropping them.
(Mark Miller)
- SOLR-8075: Leader Initiated Recovery should not stop a leader that participated in an election with all
of it's replicas from becoming a valid leader.
(Mark Miller)
- SOLR-8072: Rebalance leaders feature does not set CloudDescriptor#isLeader to false when bumping leaders.
(Mark Miller)
- SOLR-7666: Many small fixes to Angular UI
(Upayavira, Alexandre Rafalovitch)
- SOLR-7967: AddSchemaFieldsUpdateProcessorFactory does not check if the ConfigSet is immutable
(Gregory Chanan)
- SOLR-6188: Skip the automatic loading of resources in the "lib" subdirectory
by SolrResourceLoader, but only if we are loading resources from the solr
home directory. Fixes the inability to use ICU analysis components with a
"solr." prefix on the classname.
(Shawn Heisey)
- SOLR-8130: Solr's hdfs safe mode detection does not catch all cases of being in safe mode.
(Mark Miller, Mike Drob)
- SOLR-8128: Set v.locale specified locale for all LocaleConfig extending VelocityResponseWriter tools.
(Erik Hatcher)
- SOLR-8152: Overseer Task Processor/Queue can miss responses, leading to timeouts.
(Gregory Chanan)
- SOLR-8107: bin/solr -f should use exec to start the JVM
(Martijn Koster via Timothy Potter)
- SOLR-8050: Partial update on document with multivalued date field fails to parse date and can
also fail to remove dates in some cases.
(Burkhard Buelte, Luc Vanlerberghe, shalin)
- SOLR-8167: Authorization framework does not work with POST params
(noble)
- SOLR-8162: JmxMonitoredMap#clear triggers a query on all the MBeans thus generating lots of warnings.
(Marius Dumitru Florea, shalin)
- SOLR-7843: DataImportHandler's delta imports leak memory because the delta keys are kept in memory
and not cleared after the process is finished.
(Pablo Lozano via shalin)
- SOLR-8189: eTag calculation during HTTP Cache Validation uses unsynchronized WeakHashMap causing
threads to be stuck in runnable state.
(shalin)
- SOLR-7993: Raw json output for fields stopped working in 5.3.0 when requested fields do not include
the unique key field name.
(Bill Bell, Ryan McKinley via shalin)
- SOLR-8192: JSON Facet API allBuckets:true did not work correctly when faceting
on a multi-valued field with sub-facets / facet functions.
(yonik)
- SOLR-8206: JSON Facet API limit:0 did not always work correctly.
(yonik)
- SOLR-8126: update-<component-name> does not work if the component is only
present in solrconfig.xml
(noble)
- SOLR-8203: Stop processing updates more quickly on node shutdown. When a node
is shut down, streaming updates would continue, but new update requests would
be aborted. This can cause big update reorders that can cause replicas to
get out of sync.
(Mark Miller, yonik)
- SOLR-6406: ConcurrentUpdateSolrClient hang in blockUntilFinished. If updates are still
flowing and shutdown is called on the executor service used by ConcurrentUpdateSolrClient,
a race condition can cause that client to hang in blockUntilFinished.
(Mark Miller, yonik)
- SOLR-8215: Only active replicas should handle incoming requests against a collection
(Varun Thacker)
- SOLR-8223: Avoid accidentally swallowing OutOfMemoryError (in LeaderInitiatedRecoveryThread.java
or CoreContainer.java)
(Mike Drob via Christine Poerschke)
- SOLR-8255: MiniSolrCloudCluster needs to use a thread-safe list to keep track
of its child nodes
(Alan Woodward)
- SOLR-8254: HttpSolrCore.getCoreByCollection() can throw NPE
(Alan Woodward,
Mark Miller)
- SOLR-8262: Comment out /stream handler from sample solrconfig.xml's for security reasons
(Joel Bernstein)
- SOLR-7989: After a new leader is elected it should change it's state to ACTIVE even
if the last published state is something else if it has already registered with ZK.
(Ishan Chattopadhyaya, Mark Miller via noble)
- SOLR-8287: TrieDoubleField and TrieLongField now override toNativeType
(Ishan Chattopadhyaya via Christine Poerschke)
- SOLR-8284: JSON Facet API - fix NPEs when short form "sort:index" or "sort:count"
are used.
(Michael Sun via yonik)
- SOLR-8295: Fix NPE in collapse QParser when collapse field is missing from all docs in a segment
(hossman)
- SOLR-8280: Fixed bug in SimilarityFactory initialization that prevented SolrCoreAware factories -- such
as SchemaSimilarityFactory -- from functioning properly with managed schema features.
(hossman)
- SOLR-5971: Fix error 'Illegal character in query' when proxying request.
(Uwe Schindler, Ishan Chattopadhyaya, Eric Bus)
- SOLR-8307: Fix XXE vulnerability in MBeansHandler "diff" feature
(Erik Hatcher)
- SOLR-8073: Solr fails to start on Windows with obscure errors when using relative path.
(Alexandre Rafalovitch, Ishan Chattopadhyaya via shalin)
- SOLR-7169: bin/solr status should return exit code 3, not 0 if Solr is not running
(Dominik Siebel via Timothy Potter)
- SOLR-8341: Fix JSON Facet API excludeTags when specified in the
form of domain:{excludeTags:mytag}
(yonik)
- SOLR-8326: If BasicAuth enabled, inter node requests fail after node restart
(noble, Anshum Gupta)
- SOLR-8340: Fixed NullPointerException in HighlightComponent.
(zengjie via Christine Poerschke)
- SOLR-8355: update permissions were failing node recovery
(noble , Anshum Gupta)
- Optimizations (7)
- SOLR-7918: Filter (DocSet) production from term queries has been optimized and
is anywhere from 20% to over 100% faster and produces less garbage on average.
(yonik)
- SOLR-6760: New optimized DistributedQueue implementation for overseer increases
message processing performance by ~470%.
(Noble Paul, Scott Blum, shalin)
- SOLR-6629: Watch /collections zk node on all nodes so that cluster state updates
are more efficient especially when cluster has a mix of collections in stateFormat=1
and stateFormat=2.
(Scott Blum, shalin)
- SOLR-7971: Reduce memory allocated by JavaBinCodec to encode small strings by an amount
equal to the string.length(). JavaBinCodec now uses a double pass approach to write strings
larger than 64KB to avoid allocating buffer memory equal to string's UTF8 size.
(yonik, Steve Rowe, Mikhail Khludnev, Noble Paul, shalin)
- SOLR-7983: Utils.toUTF8 uses larger buffer than necessary for holding UTF8 data.
(shalin)
- SOLR-8222: JSON Facet API optimization to faceting by count on docvalue fields (or indexed fields
with method=dv) when there are multiple hits expected for enoug buckets. For example, this
more than doubled the performance of faceting 5M documents over a field with 1M unique values.
(yonik)
- SOLR-8288: DistributedUpdateProcessor#doFinish should explicitly check and ensure it
does not try to put itself into LIR.
(Mark Miller)
- Other Changes (52)
- SOLR-8294: Cleanup solrconfig.xmls under solr/example/example-DIH/solr (removed
obsolete clustering handler sections).
(Dawid Weiss)
- SOLR-7969: Unavailable clustering engines should not fail the core.
(Dawid Weiss)
- SOLR-7790, SOLR-7791: Update Carrot2 clustering component to
version 3.10.4. Upgrade HPPC library to version 0.7.1.
(Dawid Weiss)
- SOLR-7831: Start Scripts: Allow a configurable stack size [-Xss]
(Steve Davids via Mark Miller)
- SOLR-7870: Write a test which asserts that requests to stateFormat=2 collection succeed on a node
even after all local replicas of that collection have been removed.
(Scott Blum via shalin)
- SOLR-7902: Split out use of child timers from RTimer to a sub-class
(Ramkumar Aiyengar)
- SOLR-7943: Upgrade Jetty to 9.2.13.v20150730.
(Bill Bell, shalin)
- SOLR-7007: DistributedUpdateProcessor now logs replay flag as boolean instead of int
(Mike Drob via Christine Poerschke)
- SOLR-7960: Start scripts now gives generic help for bin/solr -h and bin/solr --help
(janhoy)
- SOLR-7970: Factor out a SearchGroupsFieldCommandResult class.
(Christine Poerschke)
- SOLR-7942: Previously removed unlockOnStartup option (LUCENE-6508) now logs warning if configured,
will be an error in 6.0. Also improved error msg if an index is locked on startup
(hossman)
- SOLR-7979: Fix two typos (in a CoreAdminHandler log message and a TestCloudPivotFacet comment).
(Mike Drob via Christine Poerschke)
- SOLR-7966: Solr Admin UI Solr now sets the HTTP header X-Frame-Options to DENY
to avoid clickjacking.
(yonik)
- SOLR-7999: SolrRequestParser tests no longer depend on external URLs
that may fail to work.
(Uwe Schindler)
- SOLR-8034: Leader no longer puts replicas in recovery in case of a failed update, when minRF
isn't achieved.
(Jessica Cheng, Timothy Potter, Anshum Gupta)
- SOLR-8066: SolrCore.checkStale method doesn't restore interrupt status.
(shalin)
- SOLR-8068: Throw a SolrException if the core container has initialization errors or is
shutting down
(Ishan Chattopadhyaya, Noble Paul, Anshum Gupta)
- SOLR-8083: Convert the ZookeeperInfoServlet to a handler at /admin/zookeeper
(noble)
- SOLR-8025: remove unnecessary ResponseBuilder.getQueryCommand() calls
(Christine Poerschke)
- SOLR-8150: Fix build failure due to too much output from QueryResponseTest
(janhoy)
- SOLR-8151: OverseerCollectionMessageHandler was logging info data as WARN
(Alan Woodward)
- SOLR-8116: SearchGroupsResultTransformer tweaks (String literals, list/map initialCapacity)
(Christine Poerschke)
- SOLR-8114: in Grouping.java rename groupSort and sort to withinGroupSort and groupSort
(Christine Poerschke)
- SOLR-8074: LoadAdminUIServlet directly references admin.html
(Mark Miller, Upayavira)
- SOLR-8195: IndexFetcher download trace now includes bytes-downloaded[-per-second]
(Christine Poerschke)
- SOLR-4854: Add a test to assert that [elevated] DocTransfer works correctly with javabin
response format.
(Ray, shalin)
- SOLR-8196: TestMiniSolrCloudCluster.testStopAllStartAll case plus necessary
MiniSolrCloudCluster tweak
(Christine Poerschke)
- SOLR-8221: MiniSolrCloudCluster should create subdirectories for its nodes
(Alan Woodward)
- SOLR-8218: DistributedUpdateProcessor (initialCapacity) tweaks
(Christine Poerschke)
- SOLR-8147: contrib/analytics FieldFacetAccumulator now throws IOException instead of SolrException
(Scott Stults via Christine Poerschke)
- SOLR-8239: Added ClassicSimilarityFactory, marked DefaultSimilarityFactory as deprecated.
(hossman)
- SOLR-8253: AbstractDistribZkTestBase can sometimes fail to shut down its
ZKServer
(Alan Woodward)
- SOLR-8260: Use NIO2 APIs in core discovery
(Alan Woodward)
- SOLR-8259: Deprecate JettySolrRunner.getDispatchFilter(), add
.getSolrDispatchFilter() and .getCoreContainer()
(Alan Woodward)
- SOLR-8278: Use NIO2 APIs in ConfigSetService
(Alan Woodward)
- SOLR-8286: Remove instances of solr.hdfs.blockcache.write.enabled from tests
and docs
(Gregory Chanan)
- SOLR-8269: Upgrade commons-collections to 3.2.2. This fixes a known serialization vulnerability
(janhoy)
- SOLR-8246: Fix SolrCLI to clean the config directory in case creating a core failed.
(Jason Gerlowski via Shai Erera)
- SOLR-8290: remove SchemaField.checkFieldCacheSource's unused QParser argument
(Christine Poerschke)
- SOLR-8300: Use constants for the /overseer_elect znode
(Varun Thacker)
- SOLR-8283: factor out StrParser from QueryParsing.StrParser and SortSpecParsing[Test]
from QueryParsing[Test]
(Christine Poerschke)
- SOLR-8298: small preferLocalShards implementation refactor
(Christine Poerschke)
- SOLR-8315: Removed default core checks in the dispatch filter since we don't have a default
core anymore
(Varun Thacker)
- SOLR-8302: SolrResourceLoader now takes a Path as its instance directory
(Alan
Woodward, Shawn Heisey)
- SOLR-8303: CustomBufferedIndexInput now includes resource description when
throwing EOFException.
(Mike Drob via Uwe Schindler)
- SOLR-8194: Improve error reporting for null documents in UpdateRequest
(Markus
Jelsma, Alan Woodward)
- SOLR-8277: (Search|Top)GroupsFieldCommand tweaks
(Christine Poerschke)
- SOLR-8299: ConfigSet DELETE operation no longer allows deletion of config sets that
are currently in use by other collections
(Anshum Gupta)
- SOLR-8101: Improve Linux service installation script
(Sergey Urushkin via Timothy Potter)
- SOLR-8180: jcl-over-slf4j should have officially been a SolrJ dependency; it now is.
(David Smiley, Kevin Risden)
- SOLR-8330: Standardize and fix logger creation and usage so that they aren't shared
across source files.(Jason Gerlowski, Uwe Schindler, Anshum Gupta)
- SOLR-8363: Fix check-example-lucene-match-version Ant task and addVersion.py script to
check and update luceneMatchVersion under solr/example/ configs as well logic.
(Varun Thacker)
- Bug Fixes (11)
- SOLR-8460: /analysis/field could throw exceptions for custom attributes.
(David Smiley, Uwe Schindler)
- SOLR-8373: KerberosPlugin: Using multiple nodes on same machine leads clients to
fetch TGT for every request
(Ishan Chattopadhyaya via noble)
- SOLR-8340: Fixed NullPointerException in HighlightComponent.
(zengjie via Christine Poerschke)
- SOLR-8059: &debug=results for distributed search when distrib.singlePass (sometimes activated
automatically) could result in an NPE.
(David Smiley, Markus Jelsma)
- SOLR-8167: Authorization framework does not work with POST params
(noble)
- SOLR-8355: update permissions were failing node recovery
(noble , Anshum Gupta)
- SOLR-8326: If BasicAuth enabled, inter node requests fail after node restart
(noble, Anshum Gupta)
- SOLR-8269: Upgrade commons-collections to 3.2.2. This fixes a known serialization vulnerability
(janhoy)
- SOLR-8422: When authentication enabled, requests fail if sent to a node that doesn't host
the collection
(noble)
- SOLR-7462: AIOOBE in RecordingJSONParser
(Scott Dawson, noble)
- SOLR-8496: SolrIndexSearcher.getDocSet(List<Query>) incorrectly included deleted documents
when all of the queries were uncached (or there was no filter cache). This caused
multi-select faceting (including the JSON Facet API) to include deleted doc counts
when the remaining non-excluded filters were all uncached. This bug was first introduced in 5.3.0
(Andreas Müller, Vasiliy Bout, Erick Erickson, Shawn Heisey, Hossman, yonik)
- New Features (1)
- SOLR-8470: Make TTL of PKIAuthenticationPlugin's tokens configurable through a system property
(pkiauth.ttl)
(noble)
- Bug Fixes (10)
- SOLR-7949: Resolve XSS issue in Admin UI stats page
(David Chiu via janhoy)
- SOLR-8000: security.json is not loaded on server start
(noble)
- SOLR-8004: RuleBasedAuthorization plugin does not work for the collection-admin-edit permission
(noble)
- SOLR-7972: Fix VelocityResponseWriter template encoding issue.
Templates must be UTF-8 encoded.
(Erik Hatcher)
- SOLR-7929: SimplePostTool (also bin/post) -filetypes "*" now works properly in 'web' mode
(Erik Hatcher)
- SOLR-7978: Fixed example/files update-script.js to be Java 7 and 8 compatible.
(Erik Hatcher)
- SOLR-7988: SolrJ could not make requests to handlers with '/admin/' prefix
(noble , ludovic Boutros)
- SOLR-7990: Use of timeAllowed can cause incomplete filters to be cached and incorrect
results to be returned on subsequent requests.
(Erick Erickson, yonik)
- SOLR-8041: Fix VelocityResponseWriter's $resource.get(key,baseName,locale) to use specified locale.
(Erik Hatcher)
- SOLR-8058: Fix the exclusion filter so that collections that start with js, css, img, tpl
can be accessed.
(Upayavira, Steve Rowe, Anshum Gupta)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release
- Versions of Major Components (6)
- Apache Tika 1.7
- Carrot2 3.9.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.6
- Jetty 9.2.11.v20150529
- Upgrading from Solr 5.2 (7)
- SolrJ's CollectionAdminRequest class is now marked as abstract. Use one of its concrete
sub-classes instead.
- Solr no longer supports forcefully unlocking an index.
This is no longer supported by the underlying Lucene locking
framework. The setting in solrconfig.xml has no effect anymore.
Background: If you use native lock factory, unlocking should
not be needed, because the locks are cleared after process
shutdown automatically by the operating system. If you are
using simple lock factory (not recommended) or hdfs lock
factory, you may need to manually unlock by deleting the lock
file from filesystem / HDFS.
- The zkCredientialsProvider element in solrcloud section of solr.xml is now deprecated.
Use the correct spelling (zkCredentialsProvider) instead.
- class TransformerWithContext is deprecated . Use DocTransformer directly
- The "name" parameter in ADDREPLICA Collections API call has be deprecated. One cannot specify
the core name for a replica. See SOLR-7499 for more info.
- The ShardHandler interface has changed. The interface used to provide a
`checkDistributed` function which doubled up in purpose to determine if the
request is distributed, and to prepare for distributed requests. This unfortunately
meant that the object had to be instantiated even when the request is not
distributed. The task of initially determining if the request is distributed
is now done by SearchHandler using the distrib/shards parameters, and a
ShardHandler object is created only if the request is distributed. The interface
now has a `prepDistributed` function instead of the `checkDistributed` function,
which can then be used to prepare for the distributed request. Users with custom
ShardHandler implementations would need to modify their code to this effect.
- The system property "solr.solrxml.location" is not supported any more. Now, solr.xml is first
looked up in zookeeper, and if not found, fallback to SOLR_HOME. See SOLR-7735 for more info.
Detailed Change List
- New Features (23)
- SOLR-7724: SolrJ now supports parsing the output of the clustering component.
(Alessandro Benedetti via Dawid Weiss)
- SOLR-7389: Expose znodeVersion property for each of the collections returned for the clusterstatus
operation in the collections API
(Marius Grama via shalin)
- SOLR-7622: A DocTransformer can now request fields from the SolrIndexSearcher that are not
necessarily returned in the file SolrDocument by returning a list of fields from
DocTransformer#getExtraRequestFields
(ryan)
- SOLR-7458: Expose HDFS Block Locality Metrics via JMX
(Mike Drob via Mark Miller)
- SOLR-7676: Faceting on nested objects / Block-join faceting with the new JSON Facet API.
Example: Assuming books with nested pages and an input domain of pages, the following
will switch the domain to books before faceting on the author field:
authors:{ type:terms, field:author, domain:{toParent:"type:book"} }
(yonik)
- SOLR-7668: Add 'port' tag support in replica placement rules
(Adam McElwee, Noble Paul)
- SOLR-5886: Response for an async call is now stored in zk so that it can be returned by the REQUESTSTATUS API.
Also, the number of stored (failed and successful) responses are now restricted to 10,000 each as a safety net.
(Anshum Gupta)
- SOLR-7639: MoreLikeThis QParser now supports all options provided by the MLT Handler i.e. mintf, mindf,
minwl, maxwl, maxqt, and maxntp.
- SOLR-7182: Make the Schema-API a first class citizen of SolrJ. The new SchemaRequest and its inner
classes can be used to make requests to the Schema API.
(Sven Windisch, Marius Grama via shalin)
- SOLR-7651: New response format added wt=smile
(noble)
- SOLR-4212: SOLR-6353: Let facet queries and facet ranges hang off of pivots. Example:
facet.range={!tag=r1}price&facet.query={!tag=q1}somequery&facet.pivot={!range=r1 query=q1}category,manufacturer
(Steve Molloy, hossman, shalin)
- SOLR-7742: Support for Immutable ConfigSets
(Gregory Chanan)
- SOLR-6234: Scoring for query time join
(Mikhail Khludnev)
- SOLR-2522: new two argument option for the existing field() function; picks the min/max value of a
docValues field to use as a ValueSource: "field(field_name,min)" and "field(field_name,max)"
(hossman)
- SOLR-5882: score local parameter for block join query parser {!parent}
(Andrey Kudryavtsev, Mikhail Khludnev)
- SOLR-7799: Added includeIndexFieldFlags (backwards compatible default is true) to /admin/luke.
When there are many fields in the index, setting this flag to false can dramatically speed up requests.
(ehatcher)
- SOLR-7769: Add bin/post -p alias for -port parameter.
(ehatcher)
- SOLR-7849: Solr-managed inter-node authentication when authentication enabled
(Noble Paul)
- SOLR-7220: Nested C-style comments in queries.
(yonik)
- SOLR-7757: Improved security framework where security components can be edited/reloaded, Solr
now watches /security.json. Components can choose to make their config editable
(Noble Paul, Anshum Gupta, Ishan Chattopadhyaya)
- SOLR-7838: An authorizationPlugin interface where the access control rules are stored/managed in
ZooKeeper
(Noble Paul, Anshum Gupta, Ishan Chattopadhyaya)
- SOLR-7837: An AuthenticationPlugin which implements the HTTP BasicAuth protocol and stores credentials
securely in ZooKeeper
(Noble Paul, Anshum Gupta,Ishan Chattopadhyaya)
- SOLR-7766: support creation of a coreless collection via createNodeSet=EMPTY
(Christine Poerschke)
- Bug Fixes (36)
- SOLR-7361: Slow loading SolrCores should not hold up all other SolrCores that have finished loading from serving
requests.
(Mark Miller, Timothy Potter, Ramkumar Aiyengar)
- SOLR-4506: Clean-up old (unused) index directories in the background after initializing a new index;
previously, Solr would leave old index.yyyyMMddHHmmssSSS directories left behind after failed recoveries
in the data directory, which unnecessarily consumes disk space.
(Mark Miller, Timothy Potter)
- SOLR-7108: Change default query used by /admin/ping to not rely on other parameters such as query parser or
default field.
(ehatcher)
- SOLR-6835: ReRankQueryParserPlugin checks now whether the reRankQuery parameter is present and not empty.
(帅广应, Marius Grama via shalin)
- SOLR-7566: Search requests should return the shard name that is down.
(Marius Grama, shalin)
- SOLR-7675: Add missing _root_ field to managed-schema template so that the default data driven
config set can index nested documents by default.
(yonik)
- SOLR-7635: Limit lsof port check in bin/solr to just listening ports
(Upayavira, Ramkumar Aiyengar)
- SOLR-7091: Nested documents with unknown fields don't work in schemaless mode.
(Steve Rowe)
- SOLR-7682: Schema API: add-copy-field should accept the maxChars parameter.
(Steve Rowe)
- SOLR-7693: Fix the bin/solr -e cloud example to work if lsof is not installed
on the local machine by waiting for 10 seconds before starting the second node.
(hossman, Timothy Potter)
- SOLR-7689: ReRankQuery rewrite method can change the QueryResultKey causing cache misses.
(Emad Nashed, Yonik Seeley, Joel Bernstein)
- SOLR-7697: Schema API doesn't take class or luceneMatchVersion attributes into
account for the analyzer when adding a new field type.
(Marius Grama, Steve Rowe)
- SOLR-7679: Schema API doesn't take similarity attribute into account when adding
field types.
(Marius Grama, Steve Rowe)
- SOLR-7664: Throw correct exception (RemoteSolrException) on receiving a HTTP 413.
(Ramkumar Aiyengar, Eirik Lygre)
- SOLR-6686: facet.threads can return wrong results when using facet.prefix multiple
times on same field.
(Michael Ryan, Tim Underwood via shalin)
- SOLR-7673: Race condition in shard splitting can cause operation to hang indefinitely
or sub-shards to never become active.
(shalin)
- SOLR-7741: Add missing fields to SolrIndexerConfig.toMap
(Mike Drob, Christine Poerschke via Ramkumar Aiyengar)
- SOLR-7748: Fix bin/solr to start on IBM J9.
(Shai Erera)
- SOLR-7143: MoreLikeThis Query parser should handle multiple field names
(Jens Wille, Anshum Gupta)
- SOLR-7132: The Collections API ADDREPLICA command property.name is not reflected
in the clusterstate until after Solr restarts
(Erick Erickson)
- SOLR-7172: addreplica API fails with incorrect error msg "cannot create collection"
(Erick Erickson)
- SOLR-7705: CoreAdminHandler Unload no longer handles null core name and throws NPE
instead of a bad request error.
(John Call, Edward Ribeiro via shalin)
- SOLR-7529: CoreAdminHandler Reload throws NPE on null core name instead of a bad
request error.
(Jellyfrog, Edward Ribeiro via shalin)
- SOLR-7781: JSON Facet API: Terms facet on string/text fields with sub-facets caused
a bug that resulted in filter cache lookup misses as well as the filter cache
exceeding it's configured size.
(yonik)
- SOLR-7810: map-reduce contrib script to set classpath for convenience refers to example
rather than server.
(Mark Miller)
- SOLR-7765: Hardened the behavior of TokenizerChain when null arguments are used in constructor.
This prevents NPEs in some code paths.
(Konstantin Gribov, hossman)
- SOLR-7829: Fixed a bug in distributed pivot faceting that could result in a facet.missing=true count
which was lower then the correct count if facet.sort=index and facet.pivot.mincount > 1
(hossman)
- SOLR-7842: ZK connection loss or session expiry events should not fire config directory listeners.
(noble, shalin)
- SOLR-6357: Allow delete documents by doing a score join query.
(Mikhail Khludnev, Timothy Potter)
- SOLR-7756: Fixed ExactStatsCache and LRUStatsCache to not throw an NPE when a term is not present on a shard.
(Varun Thacker, Anshum Gupta)
- SOLR-7818: Fixed distributed stats to be calculated for all the query terms. Earlier the stats were calculated with
the terms that are present in the last shard of a distributed request.
(Varun Thacker, Anshum Gupta)
- SOLR-7866: VersionInfo caused an unhandled NPE when trying to determine the max value for the
version field.
(Timothy Potter)
- SOLR-7666 (and linked tickets): Many fixes to AngularJS Admin UI bringing it close to feature
parity with existing UI.
(Upayavira)
- SOLR-7908: SegmentsInfoRequestHandler gets a ref counted IndexWriter and does not properly release it.
(Mark Miller, shalin)
- SOLR-7921: The techproducts example fails when running in a directory that contains spaces.
(Ishan Chattopadhyaya via Timothy Potter)
- SOLR-7934: SolrCLI masks underlying cause of create collection failure.
(Timothy Potter)
- Optimizations (6)
- SOLR-7660: Avoid redundant 'exists' calls made to ZK while fetching cluster state updates.
(shalin)
- SOLR-7714: Reduce SearchHandler's use of ShardHandler objects across shards in a search,
from one for each shard and the federator, to just one for the federator.
(Christine Poerschke via Ramkumar Aiyengar)
- SOLR-7751: Minor optimizations to QueryComponent.process (reduce eager instantiations,
cache method calls)
(Christine Poerschke via Ramkumar Aiyengar)
- SOLR-7455: Terms facets with the JSON Facet API now defer calculating non-sorting stats
until a second phase, after the top N facets are found. This improves performance
proportional to the number of non-sorting statistics being calculated in addition to
the number of buckets and domain documents.
For Example: The facet request {type:terms, field:field1, facet:{x:"unique(field2)"}}
saw a 7x improvement when field1 and 1M unique terms and field2 had 1000 unique terms.
(yonik)
- SOLR-7840: ZkStateReader.updateClusterState fetches watched collections twice from ZK.
(shalin)
- SOLR-7875: Speedup SolrQueryTimeoutImpl. Avoid setting a timeout time when timeAllowed
parameter is not set.
(Tomás Fernández Löbbe)
- Other Changes (42)
- SOLR-7787: Removed fastutil and java-hll dependency, integrated HyperLogLog from java-hll
into Solr core.
(Dawid Weiss)
- SOLR-7595: Allow method chaining for all CollectionAdminRequests in Solrj.
(shalin)
- SOLR-7146: MiniSolrCloudCluster based tests can fail with ZooKeeperException NoNode for /live_nodes.
(Vamsee Yarlagadda via shalin)
- SOLR-7590: Finish and improve MDC context logging support.
(Mark Miller)
- SOLR-7599: Remove cruft from SolrCloud tests.
(shalin)
- SOLR-7636: CLUSTERSTATUS API is executed at CollectionsHandler
(noble)
- LUCENE-6508: Remove ability to forcefully unlock an index.
This is no longer supported by the underlying Lucene locking
framework.
(Uwe Schindler, Mike McCandless, Robert Muir)
- SOLR-3719: Add as-you-type "instant search" to example/files /browse.
(Esther Quansah, ehatcher)
- SOLR-7645: Remove explicitly defined request handlers from example and test solrconfig's that are
already defined implicitly, such as /admin/ping, /admin/system, and several others.
(ehatcher)
- SOLR-7603: Fix test only bug in UpdateRequestProcessorFactoryTest
(hossman)
- SOLR-7634: Upgrade Jetty to 9.2.11.v20150529
(Bill Bell, shalin)
- SOLR-7659: Rename releaseCommitPointAndExtendReserve in DirectoryFileStream
to extendReserveAndReleaseCommitPoint, and reverse the code to match.
(shalin, Shawn Heisey)
- SOLR-7624: Add correct spelling (zkCredentialsProvider) as an alternative to
zkCredientialsProvider element in solrcloud section of solr.xml.
(Xu Zhang, Per Steffensen, Ramkumar Aiyengar, Mark Miller)
- SOLR-7619: Fix SegmentsInfoRequestHandlerTest when more than one segment is created.
(Ramkumar Aiyengar, Steve Rowe)
- SOLR-7678: Switch RTimer to use nanoTime (improves accuracy of QTime, and other times
returned by Solr handlers)
(Ramkumar Aiyengar)
- SOLR-7680: Use POST instead of GET when finding versions for mismatches with
CloudInspectUtil for tests
(Ramkumar Aiyengar)
- SOLR-7662: Refactored response writing to consolidate the logic in one place
(Noble Paul)
- SOLR-7665: deprecate the class TransformerWithContext
(noble)
- SOLR-7629: Have RulesTest consider disk space limitations of where the test is
being run
(Christine Poerschke via Ramkumar Aiyengar)
- SOLR-7499: The "name" parameter in ADDREPLICA Collections API call has be deprecated. One cannot specify
the core name for a replica
(Varun Thacker, noble, Erick Erickson)
- SOLR-7711: Correct initial capacity for the list that holds the default components for the SearchHandler
(Christine Poerschke via Varun Thacker)
- SOLR-7485: Replace shards.info occurrences with ShardParams.SHARDS_INFO
(Christine Poerschke via Ramkumar Aiyengar)
- SOLR-7710: Replace async occurrences with CommonAdminParams.ASYNC
(Christine Poerschke, Ramkumar Aiyengar)
- SOLR-7712: fixed test to account for aggregate floating point precision loss
(hossman)
- SOLR-7740: Fix typo bug with TestConfigOverlay
(Christine Poerschke via Ramkumar Aiyengar)
- SOLR-7750: Change TestConfig.testDefaults to cover all SolrIndexConfig fields
(Christine Poerschke via Ramkumar Aiyengar)
- SOLR-7703: Authentication plugin is now loaded using the ResourceLoader.
(Avi Digmi via Anshum Gupta)
- SOLR-7800: JSON Facet API: the avg() facet function now skips missing values
rather than treating them as a 0 value. The def() function can be used to
treat missing values as 0 if that is desired.
Example: facet:{ mean:"avg(def(myfield,0))" }
- SOLR-7805: Update Kite Morphlines to 1.1.0
(Mark Miller)
- SOLR-7803: Prevent class loading deadlock in TrieDateField; refactor date
formatting and parsing out of TrieDateField and move to static utility class
DateFormatUtil.
(Markus Heiden, Uwe Schindler)
- SOLR-7825: Forbid all usages of log4j and java.util.logging classes in Solr except
classes which are specific to logging implementations. Remove accidental usage of log4j
logger from a few places. The default log level for org.apache.zookeeper is changed from
ERROR to WARN for zkcli.{sh,cmd} only.
(Oliver Schrenk, Tim Potter, Uwe Schindler, shalin)
- SOLR-7227: Ship Solr with the Web application directory exploded into
server/solr-webapp, solr.war is no longer included in the distribution
bundles.
(Timothy Potter, Uwe Schindler)
- SOLR-6625: Enable registering interceptors for the calls made using HttpClient and make the
request object available at the interceptor context
( Ishan Chattopadhyaya, Gregory Chanan, noble, Anshum Gupta)
- SOLR-5022: On Java 7 raise permgen for running tests.
(Uwe Schindler)
- SOLR-7823: TestMiniSolrCloudCluster.testCollectionCreateSearchDelete async collection-creation (sometimes)
(Christine Poerschke)
- SOLR-7735: Look for solr.xml in Zookeeper by default in SolrCloud mode. If not found, it will be loaded
from $SOLR_HOME/solr.xml as before. Sysprop solr.solrxml.location is now gone.
(janhoy)
- SOLR-7854: Remove unused ZkStateReader.updateClusterState(false) method.
(Scott Blum via shalin)
- SOLR-7863: Lowercase the CLUSTERPROP command in ZkCLI for consistency, print error for unknown cmd
(janhoy)
- SOLR-7832: bin/post now allows either -url or -c, rather than requiring both.
(ehatcher)
- SOLR-7847: Implement run example logic in Java instead of OS-specific scripts in
bin/solr and bin\solr.cmd
(Timothy Potter)
- SOLR-7877: TestAuthenticationFramework.testBasics to preserve/restore the original request(Username|Password)
(Christine Poerschke)
- SOLR-7900: example/files improvements - added language detection and faceting, added title field, relocated .js files.
(Esther Quansah and Erik Hatcher)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release
- Versions of Major Components (6)
- Apache Tika 1.7
- Carrot2 3.9.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.6
- Jetty 9.2.10.v20150310
Detailed Change List
- Bug Fixes (9)
- SOLR-7588: Fix javascript bug introduced by SOLR-7409 that breaks the
dataimport screen in the admin UI.
(Bill Bell via Shawn Heisey)
- SOLR-7616: Faceting on a numeric field with a unique() subfacet function on another numeric field
can result in incorrect results or an exception.
(yonik)
- SOLR-7518: New Facet Module should respect shards.tolerant and process all non-failing shards
instead of throwing an exception.
(yonik)
- SOLR-7574: A request with a json content type but no body caused a null pointer exception
(yonik)
- SOLR-7512: SolrOutputFormat creates an invalid solr.xml in the solr home zip for MapReduceIndexerTool.
(Mark Miller, Adam McElwee)
- SOLR-7652: Fix example/files update-script.js to work with Java 7
(ehatcher)
- SOLR-7638: Fix new (Angular-based) admin UI Cloud pane
(Upayavira via ehatcher)
- SOLR-7655: The DefaultSolrHighlighter since 5.0 was determining if payloads were present in a way
that was slow, especially when lots of fields were highlighted. It's now fast.
(David Smiley)
- SOLR-7493: Requests aren't distributed evenly if the collection isn't present locally.
(Jeff Wartes, shalin)
- Other Changes (2)
- SOLR-7623: Fix regression from SOLR-7484 that made it impossible to override
SolrDispatchFilter#execute() and SolrDispatchFilter#sendError(). You can now override these
functions in HttpSolrCall.
(ryan)
- SOLR-7648: Expose remote IP and Host via the AuthorizationContext to be used by the authorization plugin.
(Ishan Chattopadhyaya via Anshum Gupta)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release
- Versions of Major Components (6)
- Apache Tika 1.7
- Carrot2 3.9.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.6
- Jetty 9.2.10.v20150310
- Upgrading from Solr 5.1 (6)
- A bug was introduced in Solr 4.10 that caused index time document boosts to trigger excessive field
boosts in multivalued fields -- the result being that some field norms might be excessively large.
This bug has now been fixed, but users of document boosts are strongly encouraged to re-index.
See SOLR-7335 for more details.
- The Slice and Replica classes have been changed to use State enums instead of string constants
to track the respective stats. Advanced users with client code manipulating these objects will
need to update their code accordingly. See SOLR-7325 and SOLR-7336 for more info.
- Solr has internally been upgraded to use Jetty 9. See SOLR-4839 for full details, but there are
a few key details all Solr users should know when upgrading:
-
It is no longer possible to run "java -jar start.jar" from inside the server directory.
The bin/solr script is the only supported way to run Solr. This is necessary to support
HTTP and HTTPS modules in Jetty which can be selectively enabled by the bin/solr scripts.
In case you have a pressing need to run solr the old way, you can run
"java -jar start.jar --module=http" to get the same behavior as before.
-
The way SSL support is configured has been changed. Before this release,
the SOLR_SSL_OPTS property configured in solr.in.sh (linux/mac) or solr.in.cmd (windows)
was used to enable/disable SSL but starting in 5.2.0, new properties named as
SOLR_SSL_KEY_STORE, SOLR_SSL_KEY_STORE_PASSWORD, SOLR_SSL_TRUST_STORE,
SOLR_SSL_TRUST_STORE_PASSWORD, SOLR_SSL_NEED_CLIENT_AUTH and SOLR_SSL_WANT_CLIENT_AUTH
have been introduced. The bin/solr scripts configure the SOLR_SSL_OPTS property
automatically based on the above new properties.
You should *not* configure the SOLR_SSL_OPTS property directly inside solr.in.{sh,cmd}.
-
Support for SOLR_SSL_PORT property has been removed. Instead use the regular SOLR_PORT
property or specify the port while invoking the bin/solr script using the "-p" switch.
-
Furthermore, it is now possible to configure the HTTP client with
different SSL properties than the ones used for Jetty using the same files.
-
Please refer to the "Enabling SSL" section in the Solr Reference Guide for complete details.
- Support for pathPrefix has been completely removed from Solr. Since 5.0, Solr no longer officially
supports being run as a webapp but allowed users to play around with the web.xml to have a path prefix.
That would no longer be true. See SOLR-7500 for more info.
- The package structure under org.apache.solr.client.solrj.io has been changed to support
the Streaming Expression Language (SOLR-7377). Any code written with the 5.1 Streaming API will have to
be updated to reflect these changes.
- Merge Policy's "noCFSRatio" is no longer set based on <useCompoundFile> element in the indexConfig section
of solrconfig.xml. This means that Solr will start using Lucene's default for MP "noCFSRatio", with this
new default Solr will decide if a segment should use cfs or not based on the size of the segment in relation
the size of the complete index. For TieredMergePolicy for example (current default), segments will use cfs
if they are less than 10% of the index, otherwise cfs is disabled. Old values for this setting
(1.0 for useCompoundFile=true and 0.0 for useCompoundFile=false) as well as any other value can be set
inside the <mergePolicy> element in solrconfig.xml. <useCompoundFile> will only apply to newly created
segments. See SOLR-7463.
Detailed Change List
- New Features (25)
- SOLR-6637: Restore a Solr core from a backed up index.
Restore API Example -
http://localhost:8983/solr/techproducts/replication?command=restore&name=backup_name
Restore Status API Example -
http://localhost:8983/solr/techproducts/replication?command=restorestatus
(Varun Thacker, noble, shalin)
- SOLR-7241, SOLR-7263, SOLR-7279, SOLR-7300, SOLR-7396, SOLR-7397, SOLR-7492:
Admin UI - Refactoring using AngularJS. More functionality moving the Admin
UI to Angular JS
(Upayavira via Erick)
- SOLR-7372: Limit memory consumed by LRUCache with a new 'maxRamMB' config parameter.
(yonik, shalin)
- SOLR-7376: Return raw XML or JSON (in the appropriate writer) using DocumentTransformers.
?fl=id,name,json_s:[json],xml_s:[xml]
(ryan)
- SOLR-7422: Optional flatter form for the JSON Facet API via a "type" parameter:
top_authors : { type:terms, field:author, limit:5 } is equivalent to
top_authors : { terms : { field:author, limit:5 } }
(yonik)
- SOLR-7176: zkcli script can perfrom the CLUSTERPROP command without a running Solr cluster
(Hrishikesh Gadre, Per Steffensen, Noble Paul)
- SOLR-7417: JSON Facet API - unique() is now implemented for numeric and date fields.
(yonik)
- SOLR-7406: Add a new "facet.range.method" parameter to let users choose how to do range
faceting between an implementation based on filters (previous algorithm, using
"facet.range.method=filter") or DocValues ("facet.range.method=dv").
Input parameters and output of both methods are the same.
(Tomás Fernández Löbbe)
- SOLR-7473: Facet Module (Json Facet API) range faceting now supports the "mincount"
parameter in range facets to supress buckets less than that count. The default
for "mincount" remains 0 for range faceting.
Example: prices:{ type:range, field:price, mincount:1, start:0, end:100, gap:10 }
(yonik)
- SOLR-7437: Make HDFS transaction log replication factor configurable.
(Mark Miller)
- SOLR-7477: Multi-select faceting support for the Facet Module via the "excludeTags"
parameter which disregards any matching tagged filters for that facet. Example:
& q=shoes
& fq={!tag=COLOR}color:blue
& json.facet={ colors:{type:terms, field:color, excludeTags=COLOR} }
(yonik)
- SOLR-7231: DIH-TikaEntityprocessor, create lat-lon field from Metadata
(Tim Allison via Noble Paul)
- SOLR-6220: Rule Based Replica Assignment during collection, shard creation
and replica creation
(Noble Paul)
- SOLR-6968: New 'cardinality' option for stats.field, uses HyperLogLog to efficiently
estimate the cardinality of a field w/bounded RAM.
(hossman)
- SOLR-4392: Make it possible to specify AES encrypted password in dataconfig.xml
(Noble Paul)
- SOLR-7461: stats.field now supports individual local params for 'countDistinct' and 'distinctValues'.
'calcdistinct' is still supported as an alias for both options
(hossman)
- SOLR-7522: Facet Module - Implement field/terms faceting over single-valued
numeric fields.
(yonik)
- SOLR-7275: Authorization framework for Solr. It defines an interface and a mechanism to create,
load, and use an Authorization plugin.
(Noble Paul, Ishan Chattopadhyaya, Anshum Gupta)
- SOLR-7377: Solr Streaming Expressions
(Dennis Gove, Joel Bernstein, Steven Bower)
- SOLR-7553: Facet Analytics Module: new "hll" function that uses HyperLogLog to calculate
distributed cardinality. The original "unique" function is still available.
Example: json.facet={ numProducts : "hll(product_id)" }
(yonik)
- SOLR-7546: bin/post (and SimplePostTool in -Dauto=yes mode) now sends rather than skips files
without a known content type, as "application/octet-stream", provided it still is in the
allowed filetypes setting.
(ehatcher)
- SOLR-7274: Pluggable authentication module in Solr. This defines an interface and a mechanism to create,
load, and use an Authentication plugin.
(Noble Paul, Ishan Chattopadhyaya, Gregory Chanan, Anshum Gupta)
- SOLR-7379: (experimental) New spatial RptWithGeometrySpatialField, based on CompositeSpatialStrategy,
which blends RPT indexes for speed with serialized geometry for accuracy. Includes a Lucene segment based
in-memory shape cache.
(David Smiley)
- SOLR-7465, SOLR-7610: New file indexing example, under example/files.
(Esther Quansah, Erik Hatcher)
- SOLR-7468: Kerberos authenticaion plugin for Solr. This would allow running a Kerberized Solr.
(Noble Paul, Ishan Chattopadhyaya, Gregory Chanan, Anshum Gupta)
- Bug Fixes (38)
- SOLR-6709: Fix QueryResponse to deal with the "expanded" section when using the XMLResponseParser
(Varun Thacker, Joel Bernstein)
- SOLR-7066: autoAddReplicas feature has bug when selecting replacement nodes.
(Mark Miller)
- SOLR-7370: FSHDFSUtils#recoverFileLease tries to recover the lease every one second after
the first four second wait.
(Mark Miller)
- SOLR-7369: AngularJS UI insufficient URLDecoding in cloud/tree view
(janhoy)
- SOLR-7380: SearchHandler should not try to load runtime components in inform()
(Noble Paul)
- SOLR-7385: The clusterstatus API now returns the config set used to create a collection
inside a 'configName' key.
(Shai Erera, shalin)
- SOLR-7401: Fixed a NullPointerException when concurrently creating and deleting collections,
while accessing other collections.
(Shai Erera)
- SOLR-7412: Fixed range.facet.other parameter for distributed requests.
(Will Miller, Tomás Fernándes Löbbe)
- SOLR-6087: SolrIndexSearcher makes no DelegatingCollector.finish() call when IndexSearcher
throws an expected exception.
(Christine Poerschke via shalin)
- SOLR-7420: Overseer stats are not reset on loss of ZK connection.
(Jessica Cheng, shalin)
- SOLR-7392: Fix SOLR_JAVA_MEM and SOLR_OPTS customizations in solr.in.sh being ignored
(Ramkumar Aiyengar, Ere Maijala)
- SOLR-7426: SolrConfig#getConfigOverlay does not clean up it's resources.
(Mark Miller)
- SOLR-6665: ZkController.publishAndWaitForDownStates can return before all local cores are
marked as 'down' if multiple replicas with the same core name exist in the cluster.
(shalin)
- SOLR-7418: Check and raise a SolrException instead of an NPE when an invalid doc id is sent
to the MLTQParser.
(Anshum Gupta)
- SOLR-7443: Implemented range faceting over date fields in the new facet module
(JSON Facet API).
(yonik)
- SOLR-7440: DebugComponent does not return the right requestPurpose for pivot facet refinements.
(shalin)
- SOLR-7408: Listeners set by SolrCores on config directories in ZK could be removed if collections
are created/deleted in paralle against the same config set.
(Shai Erera, Anshum Gupta)
- SOLR-7450: Fix edge case which could cause `bin/solr stop` to hang forever
(Ramkumar Aiyengar)
- SOLR-7157: initParams must support tags other than appends, defaults and, invariants
(Noble Paul)
- SOLR-7387: Facet Module - distributed search didn't work when sorting terms
facet by min, max, avg, or unique functions.
(yonik)
- SOLR-7469: Fix check-licenses to correctly detect if start.jar.sha1 is incorrect
(hossman)
- SOLR-7449: solr/server/etc/jetty-https-ssl.xml hard codes the key store file and password rather
than pulling them from the sysprops defined in solr/bin/solr.in.{sh,cmd}
- SOLR-7470: Fix sample data to eliminate file order dependency for successful indexing, also
fixed SolrCloudExampleTest to help catch this in the future.
(hossman)
- SOLR-7478: UpdateLog#close shuts down it's executor with interrupts before running it's close logic,
possibly preventing a clean close.
(Mark Miller)
- SOLR-7494: Facet Module - unique() facet function was wildly inaccurate for high cardinality
fields.
(Andy Crossen, yonik)
- SOLR-7502: start script should not try to create configset for .system collection
(Noble Paul)
- SOLR-7514: SolrClient.getByIds fails with ClassCastException
(Tom Farnworth, Ramkumar Aiyengar)
- SOLR-7531: config API shows a few keys merged together
(Noble Paul)
- SOLR-7542: Schema API: Can't remove single dynamic copy field directive
(Steve Rowe)
- SOLR-7472: SortingResponseWriter does not log fl parameters that don't exist.
(Joel Bernstein)
- SOLR-7545: Honour SOLR_HOST parameter with bin/solr{,.cmd}
(Ishan Chattopadhyaya via Ramkumar Aiyengar)
- SOLR-7503: Recovery after ZK session expiration should happen in parallel for all cores
using the thread-pool managed by ZkContainer instead of a single thread.
(Jessica Cheng Mallet, Timothy Potter, shalin, Mark Miller)
- SOLR-7335: Fix doc boosts to no longer be multiplied in each field value in multivalued fields that
are not used in copyFields
(Shingo Sasaki via hossman)
- SOLR-7585: Fix NoSuchElementException in LFUCache resulting from heavy writes
making concurrent put() calls.
(Maciej Zasada via Shawn Heisey)
- SOLR-7587: Seeding bucket versions from index when the firstSearcher event fires has a race condition
that leads to an infinite wait on VersionInfo's ReentrantReadWriteLock because the read-lock acquired
during a commit cannot be upgraded to a write-lock needed to block updates; solution is to move the
call out of the firstSearcher event path and into the SolrCore constructor.
(Timothy Potter)
- SOLR-7625: Ensure that the max value for seeding version buckets is updated after recovery even if
the UpdateLog is not replayed.
(Timothy Potter)
- SOLR-7610: Fix VelocityResponseWriter's $resource.locale to accurately report locale in use.
(ehatcher)
- SOLR-7614: Distributed pivot facet refinement was broken due to a single correlation counter
used across multiple requests as if it was private to each request.
(yonik)
- Optimizations (5)
- SOLR-7324: IndexFetcher does not need to call isIndexStale if full copy is already needed
(Stephan Lagraulet via Varun Thacker)
- SOLR-7547: Short circuit SolrDisptachFilter for static content request. Right now it creates
a new HttpSolrCall object and tries to process it.
(Anshum Gupta)
- SOLR-7333: Make the poll queue time a leader uses when distributing updates to replicas
configurable and use knowledge that a batch is being processed to poll efficiently.
(Timothy Potter)
- SOLR-7332: Initialize the highest value for all version buckets with the max value from
the index or recent updates to avoid unnecessary lookups to the index to check for reordered
updates when processing new documents.
(Timothy Potter, yonik)
- SOLR-5855: DefaultSolrHighlighter now re-uses the document's term vectors instance when highlighting
more than one field. Applies to the standard and FVH highlighters.
(David Smiley, Daniel Debray)
- Other Changes (34)
- SOLR-6865: Upgrade HttpClient to 4.4.1
(Shawn Heisey)
- SOLR-7358: TestRestoreCore fails in Windows
(Ishan Chattopadhyaya via Varun Thacker)
- SOLR-7371: Make DocSet implement Accountable to estimate memory usage.
(yonik, shalin)
- SOLR-7381: Improve logging by adding node name in MDC in SolrCloud mode and adding MDC to
all thread pools. A new MDCAwareThreadPoolExecutor is introduced and usages of
Executors#newFixedThreadPool, #newSingleThreadExecutor, #newCachedThreadPool as well as
ThreadPoolExecutor directly is now forbidden in Solr. MDC keys are now exposed in thread
names automatically so that a thread dump can give hints on what the thread was doing.
Uncaught exceptions thrown by tasks in the pool are logged along with submitter's stack trace.
(shalin)
- SOLR-7384: Fix spurious failures in FullSolrCloudDistribCmdsTest.
(shalin)
- SOLR-6692: Default highlighter changes:
-
hl.maxAnalyzedChars now applies cumulatively on a multi-valied field.
-
fragment ranking on a multi-valued field should be more relevant.
-
hl.usePhraseHighlighter is now toggleable on a per-field basis.
-
Much more extensible (get values from another source; return snippet scores and offsets).
-
When using hl.maxMultiValuedToMatch with hl.preserveMulti, only count matched snippets.
(David Smiley)
- SOLR-6886: Removed redundant size check and added missing calls to
DelegatingCollection.finish inside Grouping code.
(Christine Poerschke via shalin)
- SOLR-7421: RecoveryAfterSoftCommitTest fails frequently on Jenkins due to full index
replication taking longer than 30 seconds.
(Timothy Potter, shalin)
- SOLR-7081: Add new test case to test if create/delete/re-create collections work.
(Christine Poerschke via Ramkumar Aiyengar)
- SOLR-7467: Upgrade t-digest to 3.1
(hossman)
- SOLR-7471: Stop requiring docValues for interval faceting
(Tomás Fernández Löbbe)
- SOLR-7391: Use a time based expiration cache for one off HDFS FileSystem instances.
(Mark Miller)
- SOLR-5213: Log when shard splitting unexpectedly leads to documents going to
no or multiple shards
(Christine Poerschke, Ramkumar Aiyengar)
- SOLR-4839: Upgrade Jetty to 9.2.10.v20150310 and restlet-jee to 2.3.0
(Bill Bell, Timothy Potter, Uwe Schindler, Mark Miller, Steve Rowe, Steve Davids, shalin)
- SOLR-7425: Improve MDC based logging format.
(Mark Miller)
- SOLR-7457: Make DirectoryFactory publishing MBeanInfo extensible.
(Mike Drob via Mark Miller)
- SOLR-7325: Slice.getState() now returns a State enum instead of a String. This helps
clarify the states a Slice can be in, as well comparing the state of a Slice.
(Shai Erera)
- SOLR-7336: Added Replica.getState() and removed ZkStateReader state-related constants.
You should use Replica.State to compare a replica's state.
(Shai Erera)
- SOLR-7487: Fix check-example-lucene-match-version Ant task to check luceneMatchVersion
in solr/server/solr/configsets instead of example and harden error checking / validation
logic.
(hossman, Timothy Potter)
- SOLR-7409: When there are multiple dataimport handlers defined, the admin UI
was listing them in a random order. Now they are sorted in a natural order
that handles numbers properly.
(Jellyfrog via Shawn Heisey)
- SOLR-7484: Refactor SolrDispatchFilter to extract all Solr specific implementation detail
to HttpSolrCall and also extract methods from within the current SDF.doFilter(..) logic
making things easier to manage. HttpSolrCall converts the processing to a 3-step process
i.e. Construct, Init, and Call so the context of the request would be available after Init
and before the actual call operation.
(Anshum Gupta, Noble Paul)
- SOLR-6878: Allow symmetric lists of synonyms to be added using the managed synonym REST
API to support legacy expand=true type mappings; previously the API only allowed adding
explicit mappings, with this feature you can now add a list and have the mappings
expanded when the update is applied
(Timothy Potter, Vitaliy Zhovtyuk, hossman)
- SOLR-7102: bin/solr should activate cloud mode if ZK_HOST is set
(Timothy Potter)
- SOLR-7500: Remove pathPrefix from SolrDispatchFilter as Solr no longer runs as a part
of a bigger webapp.
(Anshum Gupta)
- SOLR-7243: CloudSolrClient was always returning SERVER_ERROR for exceptions,
even when a more relevant ErrorCode was available, via SolrException. Now
the actual ErrorCode is used when available.
(Hrishikesh Gadre via Shawn Heisey)
- SOLR-7544: CollectionsHandler refactored to be more modular
(Noble Paul)
- SOLR-7532: Removed occurrences of the unused 'commitIntervalLowerBound' property for
updateHandler elements from Solr configuration.
(Marius Grama via shalin)
- SOLR-7541: Removed CollectionsHandler#createNodeIfNotExists. All calls made to this method now call
ZkCmdExecutor#ensureExists as they were doing the same thing. Also ZkCmdExecutor#ensureExists now respects the
CreateMode passed to it.
(Varun Thacker)
- SOLR-6820: Make the number of version buckets used by the UpdateLog configurable as
increasing beyond the default 256 has been shown to help with high volume indexing
performance in SolrCloud; helps overcome a limitation where Lucene uses the request
thread to perform expensive index housekeeping work.
(Mark Miller, yonik, Timothy Potter)
- SOLR-7463: Stop forcing MergePolicy's "NoCFSRatio" based on the IWC "useCompoundFile" configuration
(Tomás Fernández Löbbe)
- SOLR-7582: Allow auto-commit to be set with system properties in data_driven_schema_configs and
enable auto soft-commits for the bin/solr -e cloud example using the Config API.
(Timothy Potter)
- SOLR-7183: Fix Locale blacklisting for Minikdc based tests.
(Ishan Chattopadhyaya, hossman
via Anshum Gupta)
- SOLR-7662: Refactored response writing to consolidate the logic in one place
(Noble Paul)
- SOLR-7110: Added option to optimize JavaBinCodec to minimize string Object creation
(Noble Paul)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release
- Versions of Major Components (6)
- Apache Tika 1.7
- Carrot2 3.9.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.6
- Jetty 8.1.10.v20130312
- Upgrading from Solr 5.0 (6)
- SolrClient query functions now declare themselves as throwing IOException in
addition to SolrServerException, to bring them in line with the update
functions.
- SolrRequest.process() is now final. Subclasses should instead be parameterized
by their corresponding SolrResponse type, and implement createResponse()
- The signature of SolrDispatchFilter.createCoreContainer() has changed to take
(String,Properties) arguments
- Deprecated the 'lib' option added to create-requesthandler as part of SOLR-6801 in 5.0 release.
Please use the add-runtimelib command
- Tika's runtime dependency of 'jhighlight' was removed as the latter was found to
contain some LGPL-only code. Until that's resolved by Tika, you can download the
.jar yourself and place it under contrib/extraction/lib.
- The _text catch-all field in data_driven_schema_configs has been renamed to _text_.
Detailed Change List
- New Features (40)
- SOLR-6909: Extract atomic update handling logic into AtomicUpdateDocumentMerger class
and enable subclassing.
(Steve Davids, yonik)
- SOLR-6845: Add a “buildOnStartup” option for suggesters.
(Tomás Fernández Löbbe)
- SOLR-6449: Add first class support for Real Time Get in Solrj.
(Anurag Sharma, Steve Davids via shalin)
- SOLR-6954: SolrClient now implements Closeable, and shutdown() has been
deprecated in favour of close().
(Mark Miller, Tomás Fernández Löbbe, Alan
Woodward)
- SOLR-4905: Allow fromIndex parameter to JoinQParserPlugin to refer to a single-sharded
collection that has a replica on all nodes where there is a replica in the to index
(Jack Lo, Timothy Potter)
- SOLR-6648: Add support in AnalyzingInfixLookupFactory and BlendedInfixLookupFactory
for setting 'highlight' and 'allTermsRequired' in the suggester configuration.
(Boon Low, Varun Thacker via Tomás Fernández Löbbe)
- SOLR-7083: Support managing all named components in solrconfig such as
requestHandler, queryParser, queryResponseWriter, valueSourceParser,
transformer, queryConverter
(Noble Paul)
- SOLR-7005: Spatial 2D heatmap faceting on RPT fields via new facet.heatmap with PNG and
2D int array formats.
(David Smiley)
- SOLR-7019: Support changing field key when using interval faceting.
(Tomás Fernández Löbbe)
- SOLR-6832: Queries be served locally rather than being forwarded to another replica.
(Sachin Goyal, Timothy Potter)
- SOLR-1945: Add support for child docs in DocumentObjectBinder
(Noble Paul, Mark Miller)
- SOLR-7125, SOLR-7158: You can upload and download configurations via CloudSolrClient
(Alan Woodward, Ishan Chattopadhyaya)
- SOLR-5507: Admin UI - Refactoring using AngularJS, first part
(Upayavira via
Erick Erickson)
- SOLR-7164: BBoxField defaults sub fields to not-stored
(ryan)
- SOLR-7155,SOLR-7201: All SolrClient methods now take an optional 'collection' argument
(Alan Woodward, Shawn Heisey)
- SOLR-6359: Allow number of logs and records kept by UpdateLog to be configured
(Ramkumar Aiyengar)
- SOLR-7189: Allow DIH to extract content from embedded documents via Tika.
(Tim Allison via shalin)
- SOLR-6841: Visualize lucene segment information in Admin UI.
(Alexey Kozhemiakin, Michal Bienkowski, hossman, Shawn Heisey, Varun Thacker via shalin)
- SOLR-5846: EnumField supports DocValues functionality.
(Elran Dvir, shalin)
- SOLR-4044: CloudSolrClient.connect() throws a more useful exception if the
cluster is not ready, and can now take an optional timeout argument to wait
for the cluster.
(Alan Woodward, shalin, yonik, Mark Miller, Vitaliy Zhovtyuk)
- SOLR-7073: Support adding a jar to a collections classpath
(Noble Paul)
- SOLR-7126: Secure loading of runtime external jars
(Noble Paul)
- SOLR-6349: Added support for stats.field localparams to enable/disable individual stats to
limit the amount of computation done and the amount of data returned.
eg: stats.field={!min=true max=true}field_name
(Tomas Fernandez-Lobbe, Xu Zhang, hossman)
- SOLR-7218: lucene/solr query syntax to give any query clause a constant score.
General Form: <clause>^=<constant_score>
Example: (color:blue color:green)^=2.0 text:shoes
(yonik)
- SOLR-7214: New Facet module with a JSON API, facet functions, aggregations, and analytics.
Any facet type can have sub facets, and facets can be sorted by arbitrary aggregation functions.
Examples:
json.facet={x:'avg(price)', y:'unique(color)'}
json.facet={count1:{query:"price:[10 TO 20]"}, count2:{query:"color:blue AND popularity:[0 TO 50]"} }
json.facet={categories:{terms:{field:cat, sort:"x desc", facet:{x:"avg(price)", y:"sum(price)"}}}}
(yonik)
- SOLR-6141: Schema API: Remove fields, dynamic fields, field types and copy
fields; and replace fields, dynamic fields and field types.
(Steve Rowe)
- SOLR-7217: HTTP POST body is auto-detected when the client is curl and the content
type is form data (curl's default), allowing users to use curl to send
JSON or XML without having to specify the content type.
(yonik)
- SOLR-6892: Update processors can now be top-level components and they can be
specified in request to create a new custom update chain
(Noble Paul)
- SOLR-7216: Solr JSON Request API:
-
HTTP search requests can have a JSON body.
-
JSON request can also be passed via the "json" parameter.
-
Smart merging of multiple JSON parameters: ruery parameters starting with "json."
will be merged into the JSON request.
-
Legacy query parameters can also be passed in the "params" block of
the JSON request.
(yonik)
- SOLR-7245: Temporary ZK election or connection loss should not stall indexing
due to leader initiated recovery
(Ramkumar Aiyengar)
- SOLR-6350: StatsComponent now supports Percentiles
(Xu Zhang, hossman)
- SOLR-7306: Percentiles support for the new facet module. Percentiles
can be calculated for all facet buckets and field faceting can sort
by percentile values.
Examples:
json.facet={ median_age : "percentile(age,50)" }
json.facet={ salary_percentiles : "percentile(salary,25,50,75)" }
(yonik)
- SOLR-7307: EmbeddedSolrServer can now be started up by passing a path to a
solr home directory, or a NodeConfig object
(Alan Woodward, Mike Drob)
- SOLR-1387: Add facet.contains and facet.contains.ignoreCase options
(Tom Winch
via Alan Woodward)
- SOLR-7082: Streaming Aggregation for SolrCloud
(Joel bernstein, Yonik Seeley)
- SOLR-7212: Parameter substitution / macro expansion across entire request.
Substitution can contain further expansions and default values are supported.
Example: q=price:[ ${low:0} TO ${high} ]&low=100&high=200
(yonik)
- SOLR-7226: Make /query/* jmx/* , requestDispatcher/*, <listener> <initParams>
properties in solrconfig.xml editable
(Noble Paul)
- SOLR-7240: '/' redirects to '/solr/' for convenience
(Martijn Koster, hossman)
- SOLR-5911: Added payload support for term vectors. New "termPayloads" option for fields
/ types in the schema, and "tv.payloads" param for the term vector component.
(Mike McCandless, David Smiley)
- SOLR-5132: Added a new collection action MODIFYCOLLECTION
(Noble Paul)
- Bug Fixes (41)
- SOLR-7046: NullPointerException when group.function uses query() function.
(Jim Musil via Erick Erickson)
- SOLR-7072: Multiple mlt.fl does not work.
(Constantin Mitocaru, shalin)
- SOLR-6775: Creating backup snapshot results in null pointer exception.
(Ryan Hesson, Varun Thacker via shalin)
- SOLR-5890: Delete silently fails if not sent to shard where document was
added
(Ishan Chattopadhyaya, Noble Paul)
- SOLR-7101: JmxMonitoredMap can throw an exception in clear when queryNames fails.
(Mark Miller, Wolfgang Hoschek)
- SOLR-6214: Snapshots numberToKeep param only keeps n-1 backups.
(Mathias H., Ramana, Varun Thacker via shalin)
- SOLR-7084: FreeTextSuggester: Better error message when doing a lookup
during dictionary build. Used to be nullpointer
(janhoy)
- SOLR-6956: OverseerCollectionProcessor and replicas on the overseer node can sometimes
operate on stale cluster state due to overseer holding the state update lock for a
long time.
(Mark Miller, shalin)
- SOLR-7104: Propagate property prefix parameters for ADDREPLICA Collections API call.
(Varun Thacker via Anshum Gupta)
- SOLR-7113: Multiple calls to UpdateLog#init is not thread safe with respect to the
HDFS FileSystem client object usage.
(Mark Miller, Vamsee Yarlagadda)
- SOLR-7128: Two phase distributed search is fetching extra fields in GET_TOP_IDS phase.
(Pablo Queixalos, shalin)
- SOLR-7139: Fix SolrContentHandler for TIKA to ignore multiple startDocument events.
(Chris A. Mattmann, Uwe Schindler)
- SOLR-7178: OverseerAutoReplicaFailoverThread compares Integer objects using ==
(shalin)
- SOLR-7171: BaseDistributedSearchTestCase now clones getSolrHome() for each subclass,
and consistently uses getSolrXml().
(hossman)
- SOLR-6657: DocumentDictionaryFactory requires weightField to be mandatory, but it shouldn't
(Erick Erickson)
- SOLR-7206: MiniSolrCloudCluster wasn't dealing with SSL mode correctly
(Alan
Woodward)
- SOLR-4464: DIH Processed documents counter resets to zero after first entity is processed.
(Dave Cook, Shawn Heisey, Aaron Greenspan, Thomas Champagne via shalin)
- SOLR-7209: /update/json/docs carry forward fields from previous records
(Noble Paul)
- SOLR-7195: Fixed a bug where the bin/solr shell script would incorrectly
detect another Solr process listening on the same port number. If the
requested listen port was 8983, it would match on another Solr using port
18983 for any purpose. Also escapes the dot character in all grep commands
looking for start.jar.
(Xu Zhang via Shawn Heisey)
- SOLR-6682: Fix response when using EnumField with StatsComponent
(Xu Zhang via hossman)
- SOLR-7109: Indexing threads stuck during network partition can put leader into down state.
(Mark Miller, Anshum Gupta, Ramkumar Aiyengar, yonik, shalin)
- SOLR-7092: Stop the HDFS lease recovery retries in HdfsTransactionLog on close and try
to avoid lease recovery on closed files.
(Mark Miller)
- SOLR-7285: ActionThrottle will not pause if getNanoTime first returns 0.
(Mark Miller, Gregory Chanan)
- SOLR-7141: RecoveryStrategy: Raise time that we wait for any updates from the leader before
they saw the recovery state to have finished.
(Mark Miller)
- SOLR-7248: In legacyCloud=false mode we should check if the core was hosted on the same node before registering it
(Varun Thacker, Noble Paul, Mark Miller)
- SOLR-7294: Migrate API fails with 'Invalid status request: notfoundretried 6times' message.
(Jessica Cheng Mallet, shalin)
- SOLR-7254: Make an invalid negative start/rows throw a HTTP 400 error (Bad Request) instead
of causing a 500 error.
(Ramkumar Aiyengar, Hrishikesh Gadre, yonik)
- SOLR-7305: BlendedInfixLookupFactory swallows root IOException when it occurs.
(Stephan Lagraulet via shalin)
- SOLR-7293: Fix bug that Solr server does not listen on IPv6 interfaces by default.
(Uwe Schindler, Sebastian Pesman)
- SOLR-7298: Fix Collections API calls (SolrJ) to not add name parameter when not needed.
(Shai Erera, Anshum Gupta)
- SOLR-7134: Replication can still cause index corruption.
(Mark Miller, shalin, Mike Drob)
- SOLR-7309: Make bin/solr, bin/post work when Solr installation directory contains spaces
(Ramkumar Aiyengar, Martijn Koster)
- SOLR-6924: The config API forcefully refreshes all replicas in the collection to ensure all are
updated
(Noble Paul)
- SOLR-7266: The IgnoreCommitOptimizeUpdateProcessor blocks commit requests from
replicas needing to recover.
(Jessica Cheng Mallet, Timothy Potter)
- SOLR-7299: bin\solr.cmd doesn't use jetty SSL configuration.
(Steve Rowe)
- SOLR-7334: Admin UI does not show "Num Docs" and "Deleted Docs".
(Erick Erickson, Timothy Potter)
- SOLR-7338, SOLR-6583: A reloaded core will never register itself as active after a ZK session expiration
(Mark Miller, Timothy Potter)
- SOLR-7366: Can't index example XML docs into the cloud example using bin/post due to regression in
ManagedIndexSchema's handling of ResourceLoaderAware objects used by field types
(Steve Rowe, Timothy Potter)
- SOLR-7284: HdfsUpdateLog is using hdfs FileSystem.get without turning off the cache.
(Mark Miller)
- SOLR-7286: Using HDFS's FileSystem.newInstance does not guarantee a new instance.
(Mark Miller)
- SOLR-7508: SolrParams.toMultiMap() does not handle arrays
(Thomas Scheffler , Noble Paul)
- Optimizations (4)
- SOLR-7049: Move work done by the LIST Collections API call to the Collections
Handler
(Varun Thacker via Anshum Gupta).
- SOLR-7116: Distributed facet refinement requests would needlessly compute other types
of faceting that have already been computed.
(David Smiley, Hossman)
- SOLR-7239: improved performance of min & max in StatsComponent, as well as situations
where local params disable all stats
(hossman)
- SOLR-7050: realtime get should internally load only fields specified in fl.
(yonik, Noble Paul)
- Other Changes (33)
- SOLR-7014: Collapse identical catch branches in try-catch statements.
(shalin)
- SOLR-6500: Refactor FileFetcher in SnapPuller, add debug logging.
(Ramkumar Aiyengar via Mark Miller)
- SOLR-7076: In DIH, TikaEntityProcessor should have support for onError=skip
(Noble Paul)
- SOLR-7094: Better error reporting of JSON parse issues when indexing docs
(Ishan Chattopadhyaya via Timothy Potter)
- SOLR-7103: Remove unused method params in faceting code.
(shalin)
- SOLR-6311: When performing distributed queries, SearchHandler should use path
when no qt or shard.qt parameter is specified; fix also resolves SOLR-4479.
(Steve Molloy, Timothy Potter)
- SOLR-7112: Fix DeleteInactiveReplicaTest.deleteLiveReplicaTest test failures.
(shalin)
- SOLR-6902: Use JUnit rules instead of inheritance with distributed Solr
tests to allow for multiple tests without the same class.
(Ramkumar Aiyengar, Erick Erickson, Mike McCandless)
- SOLR-7032: Clean up test remnants of old-style solr.xml
(Erick Erickson)
- SOLR-7145: SolrRequest is now parametrized by its response type
(Alan
Woodward)
- SOLR-7142: Fix TestFaceting.testFacets.
(Michal Kroliczek via shalin)
- SOLR-7156: Fix test failures due to resource leaks on windows.
(Ishan Chattopadhyaya via shalin)
- SOLR-7147: Introduce new TrackingShardHandlerFactory for monitoring what requests
are sent to shards during tests.
(hossman, shalin)
- SOLR-7160: Rename ConfigSolr to NodeConfig, and decouple it from xml
representation
(Alan Woodward)
- SOLR-7166: Encapsulate JettySolrRunner configuration
(Alan Woodward)
- SOLR-7130: Make stale state notification work without failing the requests
(Noble Paul, shalin)
- SOLR-7151: SolrClient query methods throw IOException
(Alan Woodward)
- SOLR-7179: JettySolrRunner no longer passes configuration to
SolrDispatchFilter via system properties, but instead uses a Properties
object in the servlet context
(Alan Woodward)
- SOLR-6275: Improve accuracy of QTime reporting
(Ramkumar Aiyengar)
- SOLR-7174: DIH should reset TikaEntityProcessor so that it is capable
of re-use
(Alexandre Rafalovitch , Gary Taylor via Noble Paul)
- SOLR-6804: Untangle SnapPuller and ReplicationHandler
(Ramkumar Aiyengar)
- SOLR-7180: MiniSolrCloudCluster will startup and shutdown its jetties in
parallel
(Alan Woodward, Tomás Fernández Löbbe, Vamsee Yarlagadda)
- SOLR-7173: Fix ReplicationFactorTest on Windows by adding better retry
support after seeing no response exceptions.
(Ishan Chattopadhyaya via Timothy Potter)
- SOLR-7246: Speed up BasicZkTest, TestManagedResourceStorage
(Ramkumar Aiyengar)
- SOLR-7258: Forbid MessageFormat.format and MessageFormat single-arg constructor.
(shalin)
- SOLR-7162: Remove unused SolrSortField interface.
(yonik, Connor Warrington via shalin)
- SOLR-6414: Update to Hadoop 2.6.0.
(Mark Miller)
- SOLR-6673: MDC based logging of collection, shard, replica, core
(Ishan Chattopadhyaya , Noble Paul)
- SOLR-7291: Test indexing on ZK disconnect with ChaosMonkey tests
(Ramkumar Aiyengar)
- SOLR-7203: Remove buggy no-op retry code in HttpSolrClient
(Alan Woodward,
Mark Miller, Greg Solovyev)
- SOLR-7202: Remove deprecated string action types in Overseer and OverseerCollectionProcessor -
"deletecollection", "createcollection", "reloadcollection", "removecollection", "removeshard".
(Varun Thacker, shalin)
- SOLR-7290: Rename catchall _text field in data_driven_schema_configs
to _text_
(Steve Rowe)
- SOLR-7346: Stored XSS in Admin UI Schema-Browser page and Analysis page
(Mei Wang via Timothy Potter)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
- NOTE: Solr 5.0 only supports creating and removing SolrCloud collections through
the collections API, unlike previous versions. While not using the
collections API may still work in 5.0, it is unsupported, not recommended,
and the behavior will change in a 5.x release.
- Versions of Major Components (6)
- Apache Tika 1.7
- Carrot2 3.9.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.6
- Jetty 8.1.10.v20130312
- Upgrading from Solr 4.x (30)
- Apache Solr has no support for Lucene/Solr 3.x and earlier indexes anymore.
Be sure to run Lucene's IndexUpgrader on the previous 4.10 version if you might
still have old segments in your index. Alternatively fully optimize your index
with Solr 4.10 to make sure it consists only of one up-to-date index segment.
- The "file" attribute of infoStream in solrconfig.xml is removed. Control this
via your logging configuration (org.apache.solr.update.LoggingInfoStream) instead.
- UniqFieldsUpdateProcessorFactory no longer supports the <lst named="fields"> init
param style that was deprecated in Solr 4.5. If you are still using this syntax,
update your configs to use <arr name="fieldName"> instead. See SOLR-4249 for more
details.
- The following legacy numeric and date field types, deprecated in Solr 4.8, are no
longer supported: BCDIntField, BCDLongField, BCDStrField, IntField, LongField,
FloatField, DoubleField, SortableIntField, SortableLongField, SortableFloatField,
SortableDoubleField, and DateField. Convert these types in your schema to the
corresponding Trie-based field type and then re-index. See SOLR-5936 for more
information.
- getAnalyzer() in IndexSchema and FieldType that was deprecated in Solr 4.9 has
been removed. Use getIndexAnalyzer() instead. See SOLR-6022 for more information.
- The spellcheck response format has changed, affecting xml and json clients. In
particular, the "correctlySpelled" and "collations" subsections have been moved outside
the "suggestions" subsection, and now are directly under "spellcheck".
See SOLR-3029 for more information.
- The CollectionsAPI SolrJ calls createCollection(), reloadCollection(),
deleteCollection(), requestStatus(), createShard(), splitShard(), deleteShard(),
createAlias() and deleteAlias() which were deprecated in 4.11 have been removed.
The new usage involves a builder style construction of the call.
- The OVERSEERSTATUS API returns new key names for operations such as "create"
for "createcollection", "delete" for "removecollection" and "deleteshard" for
"removeshard".
- If you have been using the /update/json/docs to index documents, SOLR-6617 introduces
backward incompatible change. the key names created are fully qualified paths of keys .
If you need the old functionality back , please add an extra parameter f=/**
example: /update/json/docs?f=/**
- Bugs fixed in several ValueSource functions may result in different behavior in
situations where some documents do not have values for fields wrapped in other value
sources. Users who want to preserve the previous behavior may need to wrap fields
in the "def()" function. Example: changing "fl=sum(fieldA,fieldB)" to
"fl=sum(def(fieldA,0.0),def(fieldB,0.0))". See LUCENE-5961 for more details.
- AdminHandlers is deprecated, /admin/* are implicitly defined, /get, /replication and
handlers are also implicitly registered
(refer to SOLR-6792)
- The "termIndexInterval" option in solrconfig.xml has been a No-Op in the default codec
since Solr 4.0, and has been removed completely in 5.0. If you get an "Illegal parameter
'termIndexInterval'" error when upgrading, you can safely remove this option from your
configs. If you have a strong need to configure this, you must explicitly configure your
schema with a custom codec. See SOLR-6560 and for more details.
- The "checkIntegrityAtMerge" option in solrconfig.xml is now a No-Op and should be removed
from any solrconfig.xml files -- these integrity checks are now done automatically at a very
low level during the segment merging process. See SOLR-6834 for more details.
- SimplePostTool (post.jar) no longer defaults to collection1, making either of core/collection
name or update URL mandatory. An existing call without an explicit update URL needs to now
have the core/collection name passed as "-Dc=<collection/core name>" e.g.:
java -jar -Dc=<collection_name> post.jar *.xml (new call with collection name)
See SOLR-6852 for more details.
- Relative paths specified in the solr.xml coreRootDirectory parameter for core
discovery are now resolved relative to SOLR_HOME, rather than cwd. See
SOLR-6718.
- SolrServer and associated classes have been deprecated. Applications using
SolrJ should use the equivalent SolrClient classes instead.
- Spatial fields originating from Solr 4 (e.g. SpatialRecursivePrefixTreeFieldType, BBoxField)
have the 'units' attribute deprecated, now replaced with 'distanceUnits'. If you change it to
a unit other than 'degrees' (or if you don't specify it, which will default to kilometers if
geo=true), then be sure to update maxDistErr as it's in those units. If you keep units=degrees
then it should be backwards compatible but you'll get a deprecation warning on startup. See
SOLR-6797.
- The <nrtMode> configuration in solrconfig.xml has been discontinued and should be removed from
solrconfig.xml. Solr defaults to using NRT searchers regardless of the value in configuration
and a warning is logged on startup if the solrconfig.xml has <nrtMode> specified.
- There was an old spatial syntax to specify a circle using Circle(x,y d=...) which should be
replaced with simply using {!geofilt} (if you can) or BUFFER(POINT(x y),d). Likewise a rect syntax
comprised of minX minY maxX maxY that should now be replaced with
ENVELOPE(minX, maxX, maxY, minY).
- Due to changes in the underlying commons-codec package, users of the BeiderMorseFilterFactory
will need to rebuild their indexes after upgrading. See LUCENE-6058 for more details.
- CachedSqlEntityProcessor has been removed, use SqlEntityProcessor with the
cacheImpl parameter.
- HttpDataSource has been removed, use URLDataSource instead.
- LegacyHTMLStripCharFilter has been removed
- CoreAdminRequest.persist() call has been removed. All changes made via
CoreAdmin are persistent.
- SpellCheckResponse.getSuggestions() and getSuggestionFrequencies() have been
removed, use getAlternatives() and getAlternativeFrequencies() instead.
- SolrQuery deprecated methods have been removed:
-
setMissing() is now setFacetMissing()
-
getFacetSort() is now getFacetSortString()
-
setFacetSort(boolean) should instead use setFacetSort(String) with
FacetParams.FACET_SORT_COUNT or FacetParams.FACET_SORT_INDEX
-
setSortField(String, ORDER) should use setSort(SortClause)
-
addSortField(String, ORDER) should use addSort(SortClause)
-
removeSortField(String, ORDER) should use removeSort(SortClause)
-
getSortFields() should use getSorts()
-
set/getQueryType() should use set/getRequestHandler()
- ClientUtil deprecated date methods have been removed, use DateUtil instead
- FacetParams.FacetDateOther has been removed, use FacetRangeOther
- ShardParams.SHARD_KEYS has been removed, use ShardParams._ROUTE_
- The 'old-style' solr.xml format is no longer supported, and cores must be
defined using core.properties files. See
https://cwiki.apache.org/confluence/display/solr/Format+of+solr.xml
Detailed Change List
- New Features (47)
- SOLR-6103: Added DateRangeField for indexing date ranges, especially multi-valued ones.
Supports facet.range, DateMath, and is mostly interoperable with TrieDateField.
Based on LUCENE-5648.
(David Smiley)
- SOLR-6403: TransactionLog replay status logging.
(Mark Miller)
- SOLR-4580: Support for protecting content in ZooKeeper.
(Per Steffensen, Mark Miller)
- SOLR-6365: specify appends, defaults, invariants outside of the request handler.
(Noble Paul, Erik Hatcher, shalin)
- SOLR-5097: Schema API: Add REST support for adding dynamic fields to the schema.
(Steve Rowe)
- SOLR-5098: Schema API: Add REST support for adding field types to the schema.
(Timothy Potter)
- SOLR-5473 : Split clusterstate.json per collection and watch states selectively
(Noble Paul, Mark Miller, shalin, Jessica Cheng Mallet, Timothy Potter, Anshum Gupta)
- SOLR-5474 : Support for SOLR-5473 in SolrJ
(Timothy Potter, Noble Paul, Mark Miller)
- SOLR-5810 : Support for SOLR-5473 in solr admin UI
(Timothy Potter, Noble Paul)
- SOLR-6482: Add an onlyIfDown flag for DELETEREPLICA collections API command
(Erick Erickson)
- SOLR-6354: stats.field can now be used to generate stats over the numeric results
of arbitrary functions, ie: stats.field={!func}product(price,popularity)
(hossman)
- SOLR-6485: ReplicationHandler should have an option to throttle the speed of
replication
(Varun Thacker, Noble Paul)
- SOLR-6543: Give HttpSolrClient the ability to send PUT requests
(Gregory Chanan)
- SOLR-5986: Don't allow runaway queries from harming Solr cluster health or search
performance
(Anshum Gupta, Steve Rowe, Robert Muir)
- SOLR-6565: SolrRequest support for query params
(Gregory Chanan)
- SOLR-6512: Add a collections API call to add/delete arbitrary properties
to a specific replica. Optionally adding sliceUnique=true will remove
this property from all other replicas within a particular slice.
(Erick Erickson)
- SOLR-6513: Add a collectionsAPI call BALANCESLICEUNIQUE. Allows the even
distribution of custom replica properties across nodes making up a
collection, at most one node per slice will have the property.
- SOLR-6605: Make ShardHandlerFactory maxConnections configurable.
(Christine Poerschke via shalin)
- SOLR-6585: RequestHandlers can optionally handle sub paths as well
(Noble Paul)
- SOLR-6617: /update/json/docs path will use fully qualified node names by default
(Noble Paul)
- SOLR-4715: Add CloudSolrClient constructors which accept a HttpClient instance.
(Hardik Upadhyay, Shawn Heisey, shalin)
- SOLR-5992: add "removeregex" as an atomic update operation
(Vitaliy Zhovtyuk via Erick Erickson)
- SOLR-6633: /update/json/docs path can now save the underlying json doc as a string field
and better support added to the default example
(Noble Paul)
- SOLR-6650: Add optional slow request logging at WARN level
(Jessica Cheng Mallet via Timothy Potter)
- SOLR-6655: SimplePostTool now features -Dhost, -Dport, and -Dc (for core/collection)
properties to allow easier overriding of just the right piece of the Solr URL.
(ehatcher)
- SOLR-6248: MoreLikeThis QParser that accepts a document id and returns documents that
have similar content. It works in standalone/cloud mode and shares logic with the
Lucene MoreLikeThis class
(Anshum Gupta).
- SOLR-6670: change BALANCESLICEUNIQUE to BALANCESHARDUNIQUE. Also, the parameter
for ADDREPLICAPROP that used to be sliceUnique is now shardUnique.
(Erick Erickson)
- SOLR-6351: Stats can now be nested under pivot values by adding a 'stats' local param to
facet.pivot which refers to a 'tag' local param in one or more stats.field params.
(hossman, Vitaliy Zhovtyuk, Steve Molloy)
- SOLR-6533: Support editing common solrconfig.xml values
(Noble Paul)
- SOLR-6607: Managing requesthandlers through API
(Noble Paul)
- SOLR-4799: faster join using join="zipper" aka merge join for nested DIH EntityProcessors
(Mikhail Khludnev via Noble Paul)
- SOLR-6787: API to manage blobs in Solr
(Noble Paul)
- SOLR-6801: Load RequestHandler from blob store
(Noble Paul)
- SOLR-1632: Support Distributed IDF
(Andrzej Bialecki, Mark Miller, Yonik Seeley,
Robert Muir, Markus Jelsma, Vitaliy Zhovtyuk, Anshum Gupta)
- SOLR-6729: createNodeSet.shuffle=(true|false) support for /admin/collections?action=CREATE.
(Christine Poerschke, Ramkumar Aiyengar via Mark Miller)
- SOLR-6879: Have an option to disable autoAddReplicas temporarily for all collections.
(Varun Thacker via Steve Rowe)
- SOLR-6851: Scripts to support installing and running Solr as a service on Linux
(Timothy Potter, Hossman, Steve Rowe)
- SOLR-6770: Add/edit param sets and use them in Requests
(Noble Paul)
- SOLR-6435: Add bin/post script to simplify posting content to Solr
(Erik Hatcher)
- SOLR-6761: Ability to ignore commit and/or optimize requests from clients when running in
SolrCloud mode using the IgnoreCommitOptimizeUpdateProcessorFactory.
(Timothy Potter)
- SOLR-6797: Spatial fields that used to require units=degrees like
SpatialRecursivePrefixTreeFieldType (RPT) now take distanceUnits=degrees|kilometers|miles
instead. It is applied to nearly all distance measurements involving the field: maxDistErr,
distErr, d, geodist, score=distance|area|area2d. score now accepts these units as well. It does
NOT affect distances embedded in WKT strings like BUFFER(POINT(200 10),0.2)).
(Ishan Chattopadhyaya, David Smiley)
- SOLR-6766: Expose HdfsDirectoryFactory Block Cache statistics via JMX.
(Mike Drob, Mark Miller)
- SOLR-2035: Add a VelocityResponseWriter $resource tool for locale-specific string lookups.
(Erik Hatcher)
- SOLR-6916: Toggle payload support for the default highlighter via hl.payloads. It's auto
enabled when the index has payloads.
(David Smiley)
- SOLR-6581: Efficient DocValues support and numeric collapse field implementations
for Collapse and Expand
(Joel Bernstein)
- SOLR-6937: In schemaless mode ,replace spaces and special characters with underscore
(Noble Paul)
- SOLR-5147: Support child documents in DIH
(Vadim Kirilchuk, Shawn Heisey, Thomas Champagne, Mikhail Khludnev via Noble Paul)
- Bug Fixes (65)
- SOLR-4895: An error should be returned when a rollback is attempted in SolrCloud mode.
(Vamsee Yarlagadda via Mark Miller)
- SOLR-6424: The hdfs block cache BLOCKCACHE_WRITE_ENABLED is not defaulting to false like it
should.
(Mark Miller)
- SOLR-6426: SolrZkClient clean can fail due to a race with children nodes.
(Mark Miller)
- SOLR-5966: Admin UI Menu is fixed and doesn't respect smaller viewports.
(Aman Tandon, steffkes via shalin)
- SOLR-4406: Fix RawResponseWriter to respect 'base' writer
(Steve Davids, hossman)
- SOLR-6297: Fix WordBreakSolrSpellChecker to not lose suggestions in shard/cloud
environments
(James Dyer)
- SOLR-6467: bin/solr script should direct stdout/stderr when starting in the background
to the solr-PORT-console.log in the logs directory instead of bin.
(Timothy Potter)
- SOLR-6187: SOLR-6154: facet.mincount ignored in range faceting using distributed search
NOTE: This does NOT fixed for the (deprecated) facet.date idiom, use facet.range
instead.
(Erick Erickson, Zaccheo Bagnati, Ronald Matamoros, Vamsee Yalargadda)
- SOLR-6457: LBHttpSolrClient: ArrayIndexOutOfBoundsException risk if counter overflows
(longkey via Noble Paul)
- SOLR-6499: Log warning about multiple update request handlers
(Noble Paul, Andreas Hubold, hossman)
- SOLR-6507: Fixed several bugs involving stats.field used with local params
(hossman)
- SOLR-6481: CLUSTERSTATUS should check if the node hosting a replica is live when
reporting replica status
(Timothy Potter)
- SOLR-6484: SolrCLI's healthcheck action needs to check live nodes as part of reporting
the status of a replica
(Timothy Potter)
- SOLR-6540 Fix NPE from strdist() func when doc value source does not exist in a doc
(hossman)
- SOLR-6624 Spelling mistakes in the Java source
(Hrishikesh Gadre)
- SOLR-6307: Atomic update remove does not work for int array or date array
(Anurag Sharma , noble)
- SOLR-6224: Post soft-commit callbacks are called before soft commit actually happens.
(shalin)
- SOLR-6591: Overseer can use stale cluster state and lose updates for collections
with stateFormat > 1.
(shalin)
- SOLR-6631: DistributedQueue spinning on calling zookeeper getChildren()
(Jessica Cheng Mallet, Mark Miller, Timothy Potter)
- SOLR-6579: SnapPuller Replication blocks clean shutdown of tomcat
(Philip Black-Knight via Noble Paul)
- SOLR-6721: ZkController.ensureReplicaInLeaderInitiatedRecovery puts replica
in local map before writing to ZK.
(shalin)
- SOLR-6679: Disabled suggester component from techproduct solrconfig.xml since
it caused long startup times on large indexes even when it wasn't used.
(yonik, hossman)
- SOLR-6738: Admin UI - Escape Data on Plugins-View
(steffkes)
- SOLR-3774: Solr adds RequestHandler SolrInfoMBeans twice to the JMX server.
(Tomás Fernández Löbbe, hossman, Mark Miller)
- SOLR-6763: Shard leader elections should not persist across session expiry
(Alan Woodward, Mark Miller)
- SOLR-3881: Avoid OOMs in LanguageIdentifierUpdateProcessor:
-
Added langid.maxFieldValueChars and langid.maxTotalChars params to limit
input, by default 10k and 20k chars, respectively.
-
Moved input concatenation to Tika implementation; the langdetect
implementation instead appends each input piece via the langdetect API.
(Vitaliy Zhovtyuk, Tomás Fernández Löbbe, Rob Tulloh, Steve Rowe)
- SOLR-6626: NPE in FieldMutatingUpdateProcessor when indexing a doc with
null field value
(Noble Paul)
- SOLR-6604: SOLR-6812: Fix NPE with distrib.singlePass=true and expand
component. Increased test coverage of expand component with docValues.
(Christine Poerschke, Per Steffensen, shalin)
- SOLR-6718: Core discovery was walking paths relative to the Jetty working
directory, rather than SOLR_HOME.
(Andreas Hubold, Alan Woodward)
- SOLR-6864: Support registering searcher listeners in SolrCoreAware.inform(SolrCore)
method. Existing components rely on this.
(Tomás Fernández Löbbe)
- SOLR-6850: AutoAddReplicas makes a call to wait to see live replicas that times
out after 30 milliseconds instead of 30 seconds.
(Varun Thacker via Mark Miller)
- SOLR-6873: Lib relative path is incorrect for techproduct configset
(Alexandre Rafalovitch via Erick Erickson)
- SOLR-6397: zkcli script put/putfile should allow overwriting an existing znode's data
(Timothy Potter)
- SOLR-6899: Change public setter for CollectionAdminRequest.action to protected.
(Anshum Gupta)
- SOLR-6779: fix /browse for schemaless example
(ehatcher)
- SOLR-6874: There is a race around SocketProxy binding to it's port the way we setup
JettySolrRunner and SocketProxy.
(Mark Miller, Timothy Potter)
- SOLR-6735: Make CloneFieldUpdateProcessorFactory null safe
(Steve Davids via ehatcher)
- SOLR-6907: URLEncode documents directory in MorphlineMapperTest to handle spaces etc.
in file name.
(Ramkumar Aiyengar via Erick Erickson)
- SOLR-6880: Harden ZkStateReader to expect that getCollectionLive may return null
as it's contract states.
(Mark Miller, shalin)
- SOLR-6643: Fix error reporting & logging of low level JVM Errors that occur when
loading/reloading a SolrCore
(hossman)
- SOLR-6839: Direct routing with CloudSolrServer will ignore the Overwrite document option.
(Mark Miller)
- SOLR-6793: ReplicationHandler does not destroy all of it's created SnapPullers.
(Mark Miller)
- SOLR-6946: Document -p port option for the create_core and create_collection actions in
bin/solr
(Timothy Potter)
- SOLR-6923: AutoAddReplicas also consults live_nodes to see if a state change has happened.
(Varun Thacker via Anshum Gupta)
- SOLR-6941: DistributedQueue#containsTaskWithRequestId can fail with NPE.
(Mark Miller)
- SOLR-6764: Field types need to be re-informed after reloading a managed schema from ZK
(Timothy Potter)
- SOLR-6931: We should do a limited retry when using HttpClient.
(Mark Miller, Hrishikesh Gadre, Gregory Chanan)
- SOLR-7004: Add a missing constructor for CollectionAdminRequest.BalanceShardUnique that
sets the collection action.
(Anshum Gupta)
- SOLR-6993: install_solr_service.sh won't install on RHEL / CentOS
(David Anderson via Timothy Potter)
- SOLR-6928: solr.cmd stop works only in english
(john.work, Jan Høydahl, Timothy Potter)
- SOLR-7011: Delete collection returns before collection is actually removed.
(Christine Poerschke via shalin)
- SOLR-6640: Close searchers before rollback and recovery to avoid index corruption.
(Robert Muir, Varun Thacker, shalin)
- SOLR-6847: LeaderInitiatedRecoveryThread compares wrong replica's state with lirState.
(shalin)
- SOLR-6856: Restore ExtractingRequestHandler's ability to capture all HTML tags when
parsing (X)HTML.
(hossman, Uwe Schindler, ehatcher, Steve Rowe)
- SOLR-7024: Improved error messages when java is not found by the bin/solr
shell script, particularly when JAVA_HOME has an invalid location.
(Shawn Heisey)
- SOLR-7038: Validate the presence of configset before trying to create a collection.
(Anshum Gupta, Mark Miller)
- SOLR-7037: bin/solr start -e techproducts -c fails to start Solr in cloud mode
(Timothy Potter)
- SOLR-7016: Fix bin\solr.cmd to work in a directory with spaces in the name.
(Timothy Potter, Uwe Schindler)
- SOLR-6969: When opening an HDFSTransactionLog for append we must first attempt to recover
it's lease to prevent data loss.
(Mark Miller, Praneeth Varma, Colin McCabe)
- SOLR-7067: bin/solr won't run under bash 4.2+.
(Steve Rowe)
- SOLR-7068: Collapse on numeric field breaks when min/max values are negative.
(Joel Bernstein)
- SOLR-6780: Fixed a bug in how default/appends/invariants params were affecting the set
of all "keys" found in the request parameters, resulting in some key=value param pairs
being duplicated. This was noticeably affecting some areas of the code where iteration
was done over the set of all params:
-
literal.* in ExtractingRequestHandler
-
facet.* in FacetComponent
-
spellcheck.[dictionary name].* and spellcheck.collateParam.* in SpellCheckComponent
-
olap.* in AnalyticsComponent
(Alexandre Rafalovitch & hossman)
- SOLR-6920: A replicated index can end up corrupted when small files end up with the same
file name and size.
(Varun Thacker, Mark Miller)
- SOLR-7033, SOLR-5961: RecoveryStrategy should not publish any state when
closed / cancelled and there should always be a pause between recoveries
even when recoveries are rapidly stopped and started as well as when a
node attempts to become the leader for a shard.
(Mark Miller, Maxim Novikov)
- SOLR-6693: bin\solr.cmd doesn't support 32-bit JRE/JDK running on Windows due to
parenthesis in JAVA_HOME.
(Timothy Potter, Christopher Hewitt, Jan Høydahl)
- Optimizations (4)
- SOLR-6603: LBHttpSolrClient - lazily allocate skipped-zombie-servers list.
(Christine Poerschke via shalin)
- SOLR-6554: Speed up overseer operations avoiding cluster state reads from
zookeeper at the start of each loop and instead relying on local state and
compare-and-set writes. This change also adds batching for consecutive messages
belonging to the same collection with stateFormat=2.
(shalin)
- SOLR-6680: DefaultSolrHighlighter can sometimes avoid CachingTokenFilter with
hl.usePhraseHighlighter, and can be more efficient handling data from term vectors.
(David Smiley)
- SOLR-6666: Dynamic copy fields are considering all dynamic fields, causing
a significant performance impact on indexing documents.
(Liram Vardi via Erick
Erickson, Steve Rowe)
- Other Changes (88)
- SOLR-4622: Hardcoded SolrCloud defaults for hostContext and hostPort that
were deprecated in 4.3 have been removed completely.
(hossman)
- SOLR-5936: Removed deprecated non-Trie-based numeric & date field types.
(Steve Rowe)
- SOLR-6169: Finish removal of CoreAdminHandler handleAlias action begun in 4.9
(Alan Woodward)
- SOLR-6215: TrieDateField should directly extend TrieField instead of
forwarding to a wrapped TrieField.
(Steve Rowe)
- SOLR-3029: Changes to spellcheck response format
(Nalini Kartha via James Dyer)
- SOLR-3957: Removed RequestHandlerUtils#addExperimentalFormatWarning(), which
removes "experimental" warning from two places: replication handler details
command and DataImportHandler responses.
(ehatcher)
- SOLR-6073: Remove helper methods from CollectionsRequest (SolrJ) for CollectionsAPI
calls and move to a builder design for the same.
(Varun Thacker, Anshum Gupta)
- SOLR-6519: Make DirectoryFactory#create() take LockFactory.
(Uwe Schindler)
- SOLR-6400: SolrCloud tests are not properly testing session expiration.
(Mark Miller)
- LUCENE-5650: Tests can no longer write to CWD. Update log dir is now made relative
to the instance dir if it is not an absolute path.
(Ryan Ernst, Dawid Weiss)
- SOLR-6390: Remove unnecessary checked exception for CloudSolrClient
constructors, improve javadocs for CloudSolrClient constructors.
(Steve Davids via Shawn Heisey)
- LUCENE-5901: Replaced all occurrences of LUCENE_CURRENT with LATEST for luceneMatchVersion.
(Ryan Ernst)
- SOLR-6445: Upgrade Noggit to version 0.6 to support more flexible JSON input
(Noble Paul, Yonik Seeley)
- SOLR-6073: Remove helper methods from CollectionsRequest (SolrJ) for CollectionsAPI
calls and move to a builder design for the same.
(Varun Thacker, Anshum Gupta)
- SOLR-5322: core discovery can fail w/NPE and no explanation if a non-readable directory exists
(Said Chavkin, Erick Erickson)
- SOLR-6488, SOLR-6991: Update to Apache Tika 1.7. This adds support for parsing Outlook PST and
Matlab MAT files. Parsing for NetCDF files was removed because of license issues; if you need
support for this format, download the parser JAR yourself and add it to contrib/extraction/lib
folder: http://www.unidata.ucar.edu/software/thredds/current/netcdf-java/
(Uwe Schindler)
- SOLR-6115: Cleanup enum/string action types in Overseer, OverseerCollectionProcessor and
CollectionHandler.
(Erick Erickson, shalin)
- SOLR-6453: Stop throwing an error message from Overseer when node exits
(Ramkumar Aiyengar, Noble Paul)
- SOLR-6550: Provide simple mechanism for passing additional metadata / context about a server-side
SolrException back to the client-side
(Timothy Potter)
- SOLR-6249: Schema API changes return success before all cores are updated; client application
can provide the optional updateTimeoutSecs parameter to cause the server handling the
managed schema update to block until all replicas of the same collection have processed the
update or until the specified timeout is reached
(Timothy Potter)
- SOLR-6597: SolrIndexConfig parameter in one of the SolrIndexSearcher constructor has been removed.
It was just passed and never used via that constructor.
(Anshum Gupta)
- SOLR-5852: Add CloudSolrClient helper method to connect to a ZK ensemble.
(Varun Thacker, Furkan KAMACI,
Shawn Heisey, Mark Miller, Erick Erickson via shalin)
- SOLR-6592: Avoid waiting for the leader to see the down state if that leader is not live.
(Timothy Potter)
- SOLR-6641: SystemInfoHandler should include the zkHost the node is using (when running in solrcloud mode)
(Timothy Potter)
- SOLR-6295: Fix child filter query creation to never match parent docs in SolrExampleTests.
(Varun Thacker, Mikhail Khludnev via shalin)
- SOLR-6578: Update commons-io dependency to the latest 2.4 version
(Steve Rowe, Shawn Heisey)
- SOLR-6651: Fix wrong timeout logged in waitForReplicasToComeUp.
(shalin)
- SOLR-6698: Solr is not consistent wrt ZkCredentialsProvider / ZkCredentialProvider.
References to zkCredentialProvider in System properties or configurations should be
changed to zkCredentialsProvider.
(Gregory Chanan)
- SOLR-6715: ZkSolrResourceLoader constructors accept a parameter called 'collection'
but it should be 'configName'.
(shalin)
- SOLR-6697: bin/solr start scripts allow setting SOLR_OPTS in solr.in.*
(janhoy)
- SOLR-6739: Admin UI - Sort list of command line args
(steffkes)
- SOLR-6740: Admin UI - improve Files View
(steffkes)
- SOLR-6570: Run SolrZkClient session watch asynchronously.
(Ramkumar Aiyengar via Mark Miller)
- SOLR-6747: Add an optional caching option as a workaround for SOLR-6586.
(Mark Miller, Gregory Chanan)
- SOLR-6459: Normalize logging of operations in Overseer and log current queue size.
(Ramkumar Aiyengar, shalin via Mark Miller)
- SOLR-6754: ZkController.publish doesn't use the updateLastState parameter.
(shalin)
- SOLR-6751: Exceptions thrown in the analysis chain in DirectUpdateHandler2
should return a BAD_REQUEST status
(Alan Woodward)
- SOLR-6792: deprecate AdminHandlers, Clean up solrconfig.xml of
unnecessary plugin definitions, implicit registration of /replication,
/get and /admin/* handlers
(Noble Paul)
- SOLR-4792: Stop shipping a .war.
(Robert Muir, Ramkumar Aiyengar, Mark Miller)
- SOLR-6799: Update Saxon-HE to 9.6.0-2.
(Mark Miller)
- SOLR-6454: Suppress EOFExceptions in SolrDispatchFilter.
(Ramkumar Aiyengar via Mark Miller)
- SOLR-6370: Allow tests to report/fail on many ZK watches being parallelly
requested on the same data
(Ramkumar Aiyengar via Timothy Potter)
- SOLR-6752: Buffer Cache allocate/lost metrics should be exposed.
(Mike Drob via Mark Miller)
- SOLR-6560: Purge termIndexInterval from example/test configs
(Tom Burton-West, hossman)
- SOLR-6773: Remove the multicore example as the DIH and cloud examples
illustrate multicore behavior
(hossman, Timothy Potter)
- SOLR-6834: Warn if checkIntegrityAtMerge is configured. This option is no longer meaningful
since the checks are done automatically at a very low level in the segment merging.
This warning will become an error in Solr 6.0.
(hossman)
- SOLR-6833: Examples started with bin/solr -e should use a solr.solr.home directory under
the example directory instead of server/solr.
(Alexandre Rafalovitch, Anshum Gupta, hossman,
Timothy Potter)
- SOLR-6826: fieldType capitalization is not consistent with the rest of case-sensitive field names.
(Alexandre Rafalovitch via Erick Erickson)
- SOLR-6849: HttpSolrClient.RemoteSolrException reports the URL of the remote
host where the exception occurred.
(Alan Woodward)
- SOLR-6852: SimplePostTool no longer defaults to collection1 making core/collection/update URL
mandatory.
(Anshum Gupta)
- SOLR-6861: post.sh from exampledocs directory has been removed as there no longer is a default update
URL.
(Anshum Gupta)
- SOLR-5922: Add support for adding core properties to SolrJ Collection Admin Request calls.
(Varun Thacker via Anshum Gupta).
- SOLR-6523: Provide SolrJ support for specifying stateFormat while creating Collections.
(Anshum Gupta)
- SOLR-6881: Add split.key support for SPLITSHARD via SolrJ
(Anshum Gupta)
- SOLR-6883: CLUSTERPROP API switch case does not call break.
(Varun Thacker via shalin)
- SOLR-6882: Misspelled collection API actions in ReplicaMutator exception messages.
(Steve Rowe via shalin)
- SOLR-6867: SolrCLI should check for existence before creating a new core/collection,
more user-friendly error reporting (no stack trace), and the ability to pass a
directory when using bin/solr to create a core or collection
(Timothy Potter)
- SOLR-6855: bin/solr -e dih launches, but has some path cruft issues preventing some of the
imports don't work
(Hossman, Timothy Potter)
- SOLR-3711: Truncate long strings in /browse field facets
(ehatcher)
- SOLR-6876: Remove unused legacy scripts.conf
(Alexandre Rafalovitch via Erick Erickson)
- SOLR-6896: Speed up tests by dropping SolrJettyRunner thread max idle time
(Alan Woodward)
- SOLR-6448: Add SolrJ support for all current Collection API calls.
(Anshum Gupta)
- Fixed a typo in various solrconfig.xml files.
(sdumitriu - pull request #120)
- SOLR-6895: SolrServer classes are renamed to *SolrClient. The existing
classes still exist, but are deprecated.
(Alan Woodward, Erik Hatcher)
- SOLR-6483: Refactor some methods in MiniSolrCloudCluster tests
(Steve Davids via
Erick Erickson)
- SOLR-6906: Fix typo bug in DistributedDebugComponentTest.testCompareWithNonDistributedRequest
(Ramkumar Aiyengar via Erick Erickson)
- SOLR-6905: Test pseudo-field retrieval in distributed search.
(Ramkumar Aiyengar via shalin)
- SOLR-6897: Nuke non-NRT mode from code and configuration.
(Hossman, shalin)
- SOLR-6830: Update Woodstox to 4.4.1 and StAX to 3.1.4.
(ab)
- SOLR-6918: No need to log exceptions (as warn) generated when creating MBean stats if
the core is shutting down
(Timothy Potter)
- SOLR-6932: All HttpClient ConnectionManagers and SolrJ clients should always be shutdown
in tests and regular code.
(Mark Miller)
- SOLR-1723: VelocityResponseWriter improvements
(Erik Hatcher)
- SOLR-6324: Set finite default timeouts for select and update.
(Ramkumar Aiyengar via Mark Miller)
- SOLR-6952: bin/solr create action should copy configset directory instead of reusing
an existing configset in ZooKeeper by default
(Timothy Potter)
- SOLR-6933: bin/solr should provide a single "create" action that creates a core
or collection depending on whether Solr is running in standalone or cloud mode
(Timothy Potter)
- SOLR-6496: LBHttpSolrClient stops server retries after the timeAllowed threshold is met.
(Steve Davids, Anshum Gupta)
- SOLR-6904: Removed deprecated Circle & rect syntax. See upgrading notes.
(David Smiley)
- SOLR-6943: HdfsDirectoryFactory should fall back to system props for most of it's config
if it is not found in solrconfig.xml.
(Mark Miller, Mike Drob)
- SOLR-6926: "ant example" makes no sense anymore - should be "ant server"
(Ramkumar Aiyengar, Timothy Potter)
- SOLR-6982: bin/solr and SolrCLI should support SSL-related Java System Properties
(Timothy Potter)
- SOLR-6981: Add a delete action to the bin/solr script to allow deleting of cores /
collections (with delete collection config directory from ZK)
(Timothy Potter)
- SOLR-6840: Remove support for old-style solr.xml
(Erick Erickson, Alan Woodward)
- SOLR-6976: Remove classes and methods deprecated in 4.x
(Alan Woodward, Noble
Paul, Chris Hostetter)
- SOLR-6521: CloudSolrClient should synchronize cache cluster state loading
( Noble Paul, Jessica Cheng Mallet)
- SOLR-7018: bin/solr stop should stop if there is only one node running or generate
an error message prompting the user to be explicit about which of multiple nodes
to stop using the -p or -all options
(Timothy Potter)
- SOLR-5918: ant clean does not remove ZooKeeper data
(Varun Thacker, Steve Rowe)
- SOLR-7020: 'bin/solr start' should automatically use an SSL-enabled alternate jetty
configuration file when in SSL mode, eliminating the need for manual jetty.xml edits.
(Steve Rowe)
- SOLR-6227: Avoid spurious failures of ChaosMonkeySafeLeaderTest by ensuring there's
at least one jetty to kill.
(shalin)
- Bug Fixes (23)
- SOLR-6931: We should do a limited retry when using HttpClient.
(Mark Miller, Hrishikesh Gadre, Gregory Chanan)
- SOLR-6780: Fixed a bug in how default/appends/invariants params were affecting the set
of all "keys" found in the request parameters, resulting in some key=value param pairs
being duplicated. This was noticeably affecting some areas of the code where iteration
was done over the set of all params:
-
literal.* in ExtractingRequestHandler
-
facet.* in FacetComponent
-
spellcheck.[dictionary name].* and spellcheck.collateParam.* in SpellCheckComponent
-
olap.* in AnalyticsComponent
(Alexandre Rafalovitch & hossman)
- SOLR-6426: SolrZkClient clean can fail due to a race with children nodes.
(Mark Miller)
- SOLR-6457: LBHttpSolrClient: ArrayIndexOutOfBoundsException risk if counter overflows
(longkey via Noble Paul)
- SOLR-6481: CLUSTERSTATUS should check if the node hosting a replica is live when
reporting replica status
(Timothy Potter)
- SOLR-6631: DistributedQueue spinning on calling zookeeper getChildren()
(Jessica Cheng Mallet, Mark Miller, Timothy Potter)
- SOLR-6579: SnapPuller Replication blocks clean shutdown of tomcat
(Philip Black-Knight via Noble Paul)
- SOLR-6763: Shard leader elections should not persist across session expiry
(Alan Woodward, Mark Miller)
- SOLR-3881: Avoid OOMs in LanguageIdentifierUpdateProcessor:
-
Added langid.maxFieldValueChars and langid.maxTotalChars params to limit
input, by default 10k and 20k chars, respectively.
-
Moved input concatenation to Tika implementation; the langdetect
implementation instead appends each input piece via the langdetect API.
(Vitaliy Zhovtyuk, Tomás Fernández Löbbe, Rob Tulloh, Steve Rowe)
- SOLR-6850: AutoAddReplicas makes a call to wait to see live replicas that times
out after 30 milliseconds instead of 30 seconds.
(Varun Thacker via Mark Miller)
- SOLR-6839: Direct routing with CloudSolrServer will ignore the Overwrite document option.
(Mark Miller)
- SOLR-7139: Fix SolrContentHandler for TIKA to ignore multiple startDocument events.
(Chris A. Mattmann, Uwe Schindler)
- SOLR-6941: DistributedQueue#containsTaskWithRequestId can fail with NPE.
(Mark Miller)
- SOLR-7011: Delete collection returns before collection is actually removed.
(Christine Poerschke via shalin)
- SOLR-6856: Restore ExtractingRequestHandler's ability to capture all HTML tags when
parsing (X)HTML.
(hossman, Uwe Schindler, ehatcher, Steve Rowe)
- SOLR-6928: solr.cmd stop works only in english
(john.work, Jan Høydahl, Timothy Potter)
- SOLR-7038: Validate the presence of configset before trying to create a collection.
(Anshum Gupta, Mark Miller)
- SOLR-7016: Fix bin\solr.cmd to work in a directory with spaces in the name.
(Timothy Potter, Uwe Schindler)
- SOLR-6693: bin\solr.cmd doesn't support 32-bit JRE/JDK running on Windows due to
parenthesis in JAVA_HOME.
(Timothy Potter, Christopher Hewitt, Jan Høydahl)
- SOLR-7067: bin/solr won't run under bash 4.2+.
(Steve Rowe)
- SOLR-7033, SOLR-5961: RecoveryStrategy should not publish any state when
closed / cancelled and there should always be a pause between recoveries
even when recoveries are rapidly stopped and started as well as when a
node attempts to become the leader for a shard.
(Mark Miller, Maxim Novikov)
- SOLR-6847: LeaderInitiatedRecoveryThread compares wrong replica's state with lirState.
(shalin)
- SOLR-7128: Two phase distributed search is fetching extra fields in GET_TOP_IDS phase.
(Pablo Queixalos, shalin)
- Other Changes (1)
- SOLR-7147: Introduce new TrackingShardHandlerFactory for monitoring what requests
are sent to shards during tests.
(hossman, shalin)
- Bug Fixes (19)
- SOLR-6696: bin/solr start script should not enable autoSoftCommit by default
(janhoy)
- SOLR-6704: TrieDateField type drops schema properties in branch 4.10
(Tomás Fernández Löbbe)
- SOLR-6085: Suggester crashes when prefixToken is longer than surface form
(janhoy)
- SOLR-6323: ReRankingQParserPlugin cleaner paging and fix bug with fuzzy, range and other queries
that need to be re-written.
(Adair Kovac, Joel Bernstein)
- SOLR-6684: Fix-up /export JSON.
(Joel Bernstein)
- SOLR-6781: BBoxField didn't support dynamic fields.
(David Smiley)
- SOLR-6784: BBoxField's 'score' mode should have been optional.
(David Smiley)
- SOLR-6510: The collapse QParser would throw a NPE when used on a DocValues field on
an empty segment/index.
(Christine Poerschke, David Smiley)
- SOLR-2927: Solr does not unregister all mbeans upon exception in constructor
causing memory leaks.
(tom liu, Sharath Babu, Cyrille Roy, shalin)
- SOLR-6685: ConcurrentModificationException in Overseer Status API.
(shalin)
- SOLR-6706: /update/json/docs throws RuntimeException if a nested structure
contains a non-leaf float field
(Noble Paul, shalin)
- SOLR-6610: Slow startup of new clusters because ZkController.publishAndWaitForDownStates
always times out.
(Jessica Cheng Mallet, shalin, Noble Paul)
- SOLR-6662: better validation when parsing command-line options that expect a value
(Timothy Potter)
- SOLR-6732: Fix handling of leader-initiated recovery state was String in older versions
and is now a JSON map, caused backwards compatibility issues when doing rolling upgrades of
a live cluster while indexing
(Timothy Potter)
- SOLR-6705: Better strategy for dealing with JVM specific options in the start
scripts; remove -XX:+AggressiveOpts and only set -XX:-UseSuperWord for Java 1.7u40
to u51.
(Uwe Schindler, janhoy, hossman, Timothy Potter)
- SOLR-6726: better strategy for selecting the JMX RMI port based on SOLR_PORT in bin/solr
script
(Timothy Potter)
- SOLR-6795: distrib.singlePass returns score even though not asked for.
(Per Steffensen via shalin)
- SOLR-6796: distrib.singlePass does not return correct set of fields for multi-fl-parameter
requests.
(Per Steffensen via shalin)
- SOLR-6776: Transaction log was not flushed at the end of update requests with softCommit
specified, which could lead to data loss if the server were killed immediately after the
update finished.
(Jeffery Yuan via yonik)
- Other Changes (4)
- SOLR-6661: Adjust all example configurations to allow overriding error-prone
relative paths for solrconfig.xml <lib> references with solr.install.dir
system property; bin/solr scripts will set it appropriately.
(ehatcher)
- SOLR-6694: Auto-detect JAVA_HOME using the Windows registry if it is not set
(janhoy, Timothy Potter)
- SOLR-6653: bin/solr script should return error code >0 when something fails
(janhoy, Timothy Potter)
- SOLR-6829: Added getter/setter for lastException in DIH's ContextImpl
(ehatcher)
- Bug Fixes (10)
- SOLR-6509: Solr start scripts interactive mode doesn't honor -z argument
(Timothy Potter)
- SOLR-6511: Fencepost error in LeaderInitiatedRecoveryThread
(Timothy Potter)
- SOLR-6530: Commits under network partitions can put any node in down state.
(Ramkumar Aiyengar, Alan Woodward, Mark Miller, shalin)
- SOLR-6573: QueryElevationComponent now works with localParams in the query
(janhoy)
- SOLR-6524: Collections left in recovery state after node restart because recovery sleep time
increases exponentially between retries.
(Mark Miller, shalin)
- SOLR-6587: Misleading exception when creating collections in SolrCloud with bad configuration.
(Tomás Fernández Löbbe)
- SOLR-6452: StatsComponent's stat 'missing' will work on fields with docValues=true and
indexed=false
(Xu Zhang via Tomás Fernández Löbbe)
- SOLR-6646: bin/solr start script fails to detect solr on non-default port and then after
30s tails wrong log file
(janhoy)
- SOLR-6647: Bad error message when missing resource from ZK when parsing Schema
(janhoy)
- SOLR-6545: Query field list with wild card on dynamic field fails.
(Burke Webster, Xu Zhang, shalin)
- Other Changes (4)
- SOLR-6550: Provide simple mechanism for passing additional metadata / context about a server-side
SolrException back to the client-side
(Timothy Potter)
- SOLR-6486: solr start script can have a debug flag option; use -a to set arbitrary options
(Noble Paul, Timothy Potter)
- SOLR-6549: bin/solr script should support a -s option to set the -Dsolr.solr.home property.
(Timothy Potter)
- SOLR-6529: Stop command in the start scripts should only stop the instance that it had started.
(Varun Thacker, Timothy Potter)
- Bug Fixes (6)
- SOLR-6425: If using the new global hdfs block cache option, you can end up
reading corrupt files on file name reuse.
(Mark Miller, Gregory Chanan)
- SOLR-5814: CoreContainer reports incorrect & misleading path for solrconfig.xml
when there are loading problems
(Pradeep via hossman)
- SOLR-6024: Fix StatsComponent when using docValues="true" multiValued="true"
(Vitaliy Zhovtyuk & Tomas Fernandez-Lobbe via hossman)
- SOLR-6493: Fix fq exclusion via "ex" local param in multivalued stats.field
(hossman)
- SOLR-6447: bin/solr script needs to pass -DnumShards=1 for boostrapping collection1
when starting Solr in cloud mode.
(Timothy Potter)
- SOLR-6501: Binary Response Writer does not return wildcard fields.
(Mike Hugo, Constantin Mitocaru, sarowe, shalin)
- Other Changes (1)
- SOLR-6503: Removed support for parsing netcdf files in Solr Cell because
of license issues. If you need support for this format, download the parser
JAR yourself (version 4.2) and add it to contrib/extraction/lib folder:
http://www.unidata.ucar.edu/software/thredds/current/netcdf-java/
(Uwe Schindler)
- Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release
- Versions of Major Components (5)
- Apache Tika 1.5
(with upgraded Apache POI 3.10.1)
- Carrot2 3.9.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.6
- Upgrading from Solr 4.9 (4)
- In Solr 3.6, all primitive field types were changed to omit norms by default when the
schema version is 1.5 or greater (SOLR-3140), but TrieDateField's default was mistakenly
not changed. As of Solr 4.10, TrieDateField omits norms by default (see SOLR-6211).
- Creating a SolrCore via CoreContainer.create() no longer requires an
additional call to CoreContainer.register() to make it available to clients
(see SOLR-6170).
- CoreContainer.remove() has been removed. You should now use CoreContainer.unload() to
delete a SolrCore (see SOLR-6232).
- solr.xml parsing has been improved to better account for the expected data types of
various options. As part of this fix, additional error checking has also been added to
provide errors in the event of duplicated options, or unknown option names that may
indicate a typo. Users who have modified their solr.xml in the past and now upgrade may
get errors on startup if they have typos or unexpected options specified in their solr.xml
file. (See SOLR-5746 for more information.)
Detailed Change List
- New Features (22)
- SOLR-6196: The overseerstatus collection API instruments amILeader and ZK state update calls.
(shalin)
- SOLR-6069: The 'clusterstatus' API should return 'roles' information.
(shalin)
- SOLR-6044: The 'clusterstatus' API should return live_nodes as well.
(shalin)
- SOLR-5768: Add a distrib.singlePass parameter to make EXECUTE_QUERY phase fetch all fields
and skip GET_FIELDS.
(Gregg Donovan, shalin)
- SOLR-6183: New spatial BBoxField for indexing rectangles with search support for most predicates.
It includes extra score relevancy modes in addition to distance: score=overlapRatio|area|area2D.
(David Smiley, Ryan McKinley)
- SOLR-6232: You can now unload/delete cores that have failed to initialize
(Alan Woodward)
- SOLR-2245: Improvements to the MailEntityProcessor:
-
Support for server-side date filtering if using GMail; requires new
dependency on the Sun Gmail Java mail extensions
-
Support for using the last_index_time from the previous run as the
value for the fetchMailsSince filter.
(Peter Sturge, Timothy Potter)
- SOLR-6258: Added onRollback event handler hook to Data Import Handler (DIH).
(ehatcher)
- SOLR-6263: Add DIH handler name to variable resolver as ${dih.handlerName}.
(ehatcher)
- SOLR-6216: Better faceting for multiple intervals on DV fields
(Tomas Fernandez-Lobbe
via Erick Erickson)
- SOLR-6267: Let user override Interval Faceting key with LocalParams
(Tomas Fernandez_Lobbe
via Erick Erickson)
- SOLR-6020: Auto-generate a unique key in schema-less example if data does not have an id field.
The UUIDUpdateProcessor was improved to not require a field name in configuration and generate
a UUID into the unique Key field.
(Vitaliy Zhovtyuk, hossman, Steve Rowe, Erik Hatcher, shalin)
- SOLR-6294: SOLR-6437: Remove the restriction of adding json by only wrapping it in an array in a
new path /update/json/docs
(Noble Paul , hossman, Yonik Seeley, Steve Rowe)
- SOLR-6302: UpdateRequestHandlers are registered implicitly /update ,
/update/json, /update/csv , /update/json/docs
(Noble Paul)
- SOLR-6318: New "terms" QParser for efficiently filtering documents by a list of values. For
many values, it's more appropriate than a boolean query.
(David Smiley)
- SOLR-6283: Add support for Interval Faceting in SolrJ.
(Tomás Fernández Löbbe)
- SOLR-6304 : JsonLoader should be able to flatten an input JSON to multiple docs
(Noble Paul)
- SOLR-2894: Distributed query support for facet.pivot
(Dan Cooper, Erik Hatcher, Chris Russell,
Andrew Muldowney, Brett Lucey, Mark Miller, hossman)
- SOLR-5656: Add autoAddReplicas feature for shared file systems.
(Mark Miller, Gregory Chanan)
- SOLR-5244: Exporting Full Sorted Result Sets
(Erik Hatcher, Joel Bernstein)
- SOLR-3617: bin/solr and bin/solr.cmd scripts for starting, stopping, and running Solr examples
(Timothy Potter)
- SOLR-6233: Provide basic command line tools for checking Solr status and health.
(Timothy Potter)