- Bug Fixes (38)
- SOLR-6095 : SolrCloud cluster can end up without an overseer with overseer roles
(Noble Paul, Shalin Mangar)
- SOLR-6165: DataImportHandler should write BigInteger and BigDecimal values as strings.
(Anand Sengamalai via shalin)
- SOLR-6189: Avoid publishing the state as down if the node is not live when determining
if a replica should be in leader-initiated recovery.
(Timothy Potter)
- SOLR-6197: The MIGRATE collection API doesn't work when legacyCloud=false is set
in cluster properties.
(shalin)
- SOLR-6206: The migrate collection API fails on retry if temp collection already exists.
(shalin)
- SOLR-6072: The 'deletereplica' API should remove the data and instance directory by default.
(shalin)
- SOLR-6211: TrieDateField doesn't default to omitNorms=true.
(Michael Ryan, Steve Rowe)
- SOLR-6159: A ZooKeeper session expiry during setup can keep LeaderElector from joining elections.
(Steven Bower, shalin)
- SOLR-6223: SearchComponents may throw NPE when using shards.tolerant and there is a failure
in the 'GET_FIELDS/GET_HIGHLIGHTS/GET_DEBUG' phase.
(Tomás Fernández Löbbe via shalin)
- SOLR-6180: Callers of ManagedIndexSchema mutators should hold the schemaUpdateLock.
(Gregory Chanan via Steve Rowe)
- SOLR-6229: Make SuggestComponent return 400 instead of 500 for bad dictionary selected in request.
(Tomás Fernández Löbbe via shalin)
- SOLR-6235: Leader initiated recovery should use coreNodeName instead of coreName to avoid marking
all replicas having common core name as down.
(shalin)
- SOLR-6208: JettySolrRunner QueuedThreadPool's configuration code is never executed.
(dweiss via shalin)
- SOLR-6245: Socket and Connection configuration are ignored in HttpSolrServer when passing in HttpClient.
(Patanachai Tangchaisin, shalin)
- SOLR-6137: Schemaless concurrency improvements:
-
Fixed an NPE when reloading a managed schema with no dynamic copy fields
-
Moved parsing and schema fields addition to after the distributed phase
-
AddSchemaFieldsUpdateProcessor now uses a fixed schema rather than always
retrieving the latest, and holds the schema update lock through the entire
schema swap-out process
(Gregory Chanan via Steve Rowe)
- SOLR-6136: ConcurrentUpdateSolrServer includes a Spin Lock
(Brandon Chapman, Timothy Potter)
- SOLR-6257: More than two "!"-s in a doc ID throws an
ArrayIndexOutOfBoundsException when using the composite id router.
(Steve Rowe)
- SOLR-5746: Bugs in solr.xml parsing have been fixed to more correctly deal with the various
datatypes of options people can specify, additional error handling of duplicated/unidentified
options has also been added.
(Maciej Zasada, hossman)
- SOLR-5847: Fixed data import abort button in admin UI.
(ehatcher)
- SOLR-6264: Distributed commit and optimize are executed serially across all
replicas.
(Mark Miller, Timothy Potter)
- SOLR-6163: Correctly decode special characters in managed stopwords and synonym endpoints.
(Vitaliy Zhovtyuk, Timo Schmidt via Timothy Potter)
- SOLR-6336: DistributedQueue can easily create too many ZooKeeper Watches.
(Ramkumar Aiyengar via Mark Miller)
- SOLR-6347: DELETEREPLICA throws a NPE while removing the last Replica in a Custom
sharded collection.
(Anshum Gupta)
- SOLR-6062: Fix undesirable edismax query parser effect (introduced in SOLR-2058) in how phrase queries
generated from pf, pf2, and pf3 are merged into the main query.
(Michael Dodsworth via ehatcher)
- SOLR-6372: HdfsDirectoryFactory should use supplied Configuration for communicating with secure kerberos.
(Gregory Chanan via Mark Miller)
- SOLR-6284: Fix NPE in OCP when non-existent sliceId is used for a
deleteShard request
(Ramkumar Aiyengar via Anshum Gupta)
- SOLR-6380: Added missing context info to log message if IOException occurs in processing tlog
(Steven Bower via hossman)
- SOLR-6383: RegexTransformer returns no results after replaceAll if regex does not match a value.
(Alexander Kingson, shalin)
- SOLR-6387: Add better error messages throughout Solr and supply a work around for
Java bug #8047340 to SystemInfoHandler: On Turkish default locale, some JVMs fail
to fork on MacOSX, BSD, AIX, and Solaris platforms.
(hossman, Uwe Schindler)
- SOLR-6338: coreRootDirectory requires trailing slash, or SolrCloud cores are created in wrong location.
(Primož Skale via Erick Erickson)
- SOLR-6314: Facet counts duplicated in the response if specified more than once on the request.
(Vamsee Yarlagadda, Erick Erickson)
- SOLR-6378: Fixed example/example-DIH/ issues with "tika" and "solr" configurations, and tidied up README.txt
(Daniel Shchyokin via ehatcher)
- SOLR-6393: TransactionLog replay performance on HDFS is very poor.
(Mark Miller)
- SOLR-6268: HdfsUpdateLog has a race condition that can expose a closed HDFS FileSystem instance and should
close its FileSystem instance if either inherited close method is called.
(Mark Miller)
- SOLR-6089: When using the HDFS block cache, when a file is deleted, its underlying data entries in the
block cache are not removed, which is a problem with the global block cache option.
(Mark Miller, Patrick Hunt)
- SOLR-6402: OverseerCollectionProcessor should not exit for ZooKeeper ConnectionLoss.
(Jessica Cheng via Mark Miller)
- SOLR-6405: ZooKeeper calls can easily not be retried enough on ConnectionLoss.
(Jessica Cheng, Mark Miller)
- SOLR-6410: Ensure all Lookup instances are closed via CloseHook
(hossman, Areek Zillur, Ryan Ernst, Dawid Weiss)
- Optimizations (4)
- LUCENE-5803: Solr's schema now uses DelegatingAnalyzerWrapper. This uses less heap
for cached TokenStreamComponents because it caches per FieldType not per Field, so
indexes with many fields of same type just use one TokenStream per thread.
(Shay Banon, Uwe Schindler, Robert Muir)
- SOLR-6259: Reduce CPU usage by avoiding repeated costly calls to Document.getField inside
DocumentBuilder.toDocument for use-cases with large number of fields and copyFields.
(Steven Bower via shalin)
- SOLR-5968: BinaryResponseWriter fetches unnecessary stored fields when only pseudo-fields
are requested.
(Gregg Donovan via shalin)
- SOLR-6261: Run ZooKeeper watch event callbacks in parallel to the ZooKeeper
event thread.
(Ramkumar Aiyengar via Mark Miller)
- Other Changes (33)
- SOLR-6173: Fixed wrong failure message in TestDistributedSearch.
(shalin)
- SOLR-5902: Corecontainer level mbeans are not exposed
(noble)
- SOLR-6194: Allow access to DataImporter and DIHConfiguration from DataImportHandler.
(Aaron LaBella via shalin)
- SOLR-6170: CoreContainer.preRegisterInZk() and CoreContainer.register() commands
are merged into CoreContainer.create().
(Alan Woodward)
- SOLR-6171: Remove unused SolrCores coreNameToOrig map
(Alan Woodward)
- SOLR-5596: Set system property zookeeper.forceSync=no for Solr test cases.
(shalin)
- SOLR-2853: Add a unit test for the case when "spellcheck.maxCollationTries=0"
(James Dyer)
- SOLR-6240: Removed unused coreName parameter in ZkStateReader.getReplicaProps.
(shalin)
- SOLR-6241: Harden the HttpPartitionTest.
(shalin)
- SOLR-6228: Fixed bug in TestReplicationHandler.doTestIndexAndConfigReplication.
(shalin)
- SOLR-6120: On Windows, when the war is not extracted, the zkcli.bat script
will print a helpful message indicating that the war must be unzipped instead
of a java error about a missing class.
(shalin, Shawn Heisey)
- SOLR-6179: Better strategy for handling empty managed data to avoid spurious
warning messages in the logs.
(Timothy Potter)
- SOLR-6232: CoreContainer.remove() replaced with CoreContainer.unload(). A call to
unload will also close the core.
- SOLR-3893: DIH should not depend on mail.jar,activation.jar
(Timothy Potter, Steve Rowe)
- SOLR-6252: A couple of small improvements to UnInvertedField class.
(Vamsee Yarlagadda, Gregory Chanan, Mark Miller)
- SOLR-3345: BaseDistributedSearchTestCase should always ignore QTime.
(Vamsee Yarlagadda, Benson Margulies via Mark Miller)
- SOLR-6270: Increased timeouts for MultiThreadedOCPTest.
(shalin)
- SOLR-6274: UpdateShardHandler should log the params used to configure its
HttpClient.
(Ramkumar Aiyengar via Mark Miller)
- SOLR-6194: Opened up "public" access to DataSource, DocBuilder, and EntityProcessorWrapper
in DIH.
(Aaron LaBella via ehatcher)
- SOLR-6269: Renamed "rollback" to "error" in DIH internals, including renaming onRollback
to onError introduced in SOLR-6258.
(ehatcher)
- SOLR-3622: When using DIH in SolrCloud-mode, rollback will no longer be called when
an error occurs.
(ehatcher)
- SOLR-6231: Increased timeouts and hardened the RollingRestartTest.
(Noble Paul, shalin)
- SOLR-6290: Harden and speed up CollectionsAPIAsyncDistributedZkTest.
(Mark Miller, shalin)
- SOLR-6281: Made PostingsSolrHighlighter more configurable via subclass extension.
(David Smiley)
- SOLR-6309: Increase timeouts for AsyncMigrateRouteKeyTest.
(shalin)
- SOLR-2168: Added support for facet.missing in /browse field and pivot faceting.
(ehatcher)
- SOLR-4702: Added support for multiple spellcheck collations to /browse UI.
(ehatcher)
- SOLR-5664: Added support for multi-valued field highlighting in /browse UI.
(ehatcher)
- SOLR-6313: Improve SolrCloud cloud-dev scripts.
(Mark Miller, Vamsee Yarlagadda)
- SOLR-6360: Remove bogus "Content-Charset" header in HttpSolrServer.
(Michael Ryan,
Uwe Schindler)
- SOLR-6362: Fix bug in TestSqlEntityProcessorDelta.
(James Dyer)
- SOLR-6388: Force upgrade of Apache POI dependency in Solr Cell to version
3.10.1 to fix CVE-2014-3529 and CVE-2014-3574.
(Uwe Schindler)
- SOLR-6391: Improve message for CREATECOLLECTION failure due to missing
numShards
(Anshum Gupta)
- Versions of Major Components (5)
- Apache Tika 1.5
- Carrot2 3.9.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.6
- Upgrading from Solr 4.8 (1)
- Support for DiskDocValuesFormat (ie: fieldTypes configured with docValuesFormat="Disk")
has been removed due to poor performance. If you have an existing fieldTypes using
DiskDocValuesFormat please modify your schema.xml to remove the 'docValuesFormat'
attribute, and optimize your index to rewrite it into the default codec, prior to
upgrading to 4.9. See LUCENE-5761 for more details.
Detailed Change List
- New Features (10)
- SOLR-5999: Add checkIntegrityAtMerge support to solrconfig.xml.
(Varun Thacker via Ryan Ernst)
- SOLR-6043: Add ability to set http headers in solr response
(Tomás Fernández Löbbe via Ryan Ernst)
- SOLR-5973: Pluggable Ranking Collectors and Merge Strategies
(Joel Bernstein)
- SOLR-6108: Add support for 'addreplica' Collection API in SolrJ.
(shalin)
- SOLR-5468: Allow a client application to request the minium achieved
replication factor for an update request (single or batch) by sending
an optional parameter "min_rf".
(Timothy Potter)
- SOLR-6088: Add query re-ranking with the ReRankingQParserPlugin
(Joel Bernstein)
- SOLR-5285: Added a new [child ...] DocTransformer for optionally including
Block-Join descendant documents inline in the results of a search. This works
independent of whether the search itself is a block-join related query and is
supported by he xml, json, and javabin response formats.
(Varun Thacker via hossman)
- SOLR-6150: Add new AnalyticsQuery to support pluggable analytics
(Joel Bernstein)
- SOLR-6125: Allow SolrIndexWriter to close without waiting for merges
(Christine Poerschke via Alan Woodward)
- SOLR-6064: DebugComponent track output should be returned as a JSON
object rather than a list
(Christine Poerschke, Alan Woodward)
- Bug Fixes (18)
- SOLR-5956: Use coreDescriptor.getInstanceDir() instead of getRawInstanceDir()
in the SnapShooter to avoid problems when solr.solr.home is a symbolic link.
(Timothy Potter)
- SOLR-6002: Fix a couple of ugly issues around SolrIndexWriter close and
rollback as well as how SolrIndexWriter manages its ref counted directory
instance.
(Mark Miller, Gregory Chanan)
- SOLR-6015: Better way to handle managed synonyms when ignoreCase=true
(Timothy Potter)
- SOLR-6104: The 'addreplica' Collection API does not support 'async' parameter.
(shalin)
- SOLR-6101: Shard splitting doesn't work when legacyCloud=false is set in
cluster properties.
(shalin)
- SOLR-6111: The 'deleteshard' collection API should be able to delete a shard
in 'construction' state.
(shalin)
- SOLR-6118: 'expand.sort' didn't support function queries.
(David Smiley)
- SOLR-6120: zkcli.sh should expand solr.war automatically instead of throwing
ClassNotFoundException.
(sebastian badea, shalin)
- SOLR-6149: Specifying the query value without any index value does not work in
Analysis browser.
(Aman Tandon, shalin)
- SOLR-6145: Fix Schema API optimistic concurrency by moving it out of
ManagedIndexSchema.add(Copy)Fields() into the consumers of those methods:
CopyFieldCollectionResource, FieldCollectionResource, FieldResource,
and AddSchemaFieldsUpdateProcessorFactory.
(Gregory Chanan, Alexey Serba, Steve Rowe)
- SOLR-6146: Incorrect configuration such as wrong chroot in zk server address can
cause CloudSolrServer to leak resources.
(Jessica Cheng, Varun Thacker, shalin)
- SOLR-6158: Relative configSetBase directories were resolved relative to the
container CWD, rather than solr.home.
(Simon Endele, Alan Woodward)
- SOLR-5426: Fixed a bug in ReverseWildCardFilter that could cause
InvalidTokenOffsetsException when highlighting.
(Uwe Schindler, Arun Kumar, via hossman)
- SOLR-6175: DebugComponent throws NPE on shard exceptions when using shards.tolerant.
(Tomás Fernández Löbbe via shalin)
- SOLR-6129: DateFormatTransformer doesn't resolve dateTimeFormat.
(Aaron LaBella via shalin)
- SOLR-6164: Copy Fields Schema additions are not distributed to other nodes.
(Gregory Chanan via Steve Rowe)
- SOLR-6160: An error was sometimes possible if a distributed search included grouping
with group.facet, faceting on facet.field and either facet.range or facet.query.
(David Smiley)
- SOLR-6182: Data stored by the RestManager could not be reloaded after core restart, causing
the core to fail to load; cast the data loaded from storage to the correct data type.
(Timothy Potter)
- Other Changes (19)
- SOLR-5980: AbstractFullDistribZkTestBase#compareResults always returns false
for shouldFail.
(Mark Miller, Gregory Chanan)
- SOLR-5987: Add "collection" to UpdateParams.
(Mark Miller, Greg Solovyev)
- SOLR-3862: Add remove" as update option for atomically removing a value
from a multivalued field
(Jim Musli, Steven Bower, Alaknantha via Erick Erickson)
- SOLR-5974: Remove ShardDoc.score and use parent's ScoreDoc.score.
(Tomás Fernández Löbbe via Ryan Ernst)
- SOLR-6025: Replace mentions of CommonsHttpSolrServer with HttpSolrServer and
StreamingUpdateSolrServer with ConcurrentUpdateSolrServer.
(Ahmet Arslan via shalin)
- SOLR-6013: Fix method visibility of Evaluator, refactor DateFormatEvaluator for
extensibility.
(Aaron LaBella via shalin)
- SOLR-6022: Deprecate getAnalyzer() in IndexField and FieldType, and add getIndexAnalyzer().
(Ryan Ernst)
- SOLR-3671: Fix DIHWriter interface usage so users may implement writers that output
documents to a location external to Solr (ex. a NoSql db).
(Roman Chyla via James Dyer)
- SOLR-5340: Add support for named snapshots
(Varun Thacker via Noble Paul)
- SOLR-5495: Recovery strategy for leader partitioned from replica case. Hardening
recovery scenarios after the leader receives an error trying to forward an
update request to a replica.
(Timothy Potter)
- SOLR-6116: Refactor DocRouter.getDocRouter to accept routerName as a String.
(shalin)
- SOLR-6026: REQUESTSTATUS Collection API now also checks for submitted tasks which are
yet to begin execution.
- SOLR-6067: Refactor duplicate Collector code in SolrIndexSearcher
(Christine Poerschke via hossman)
- SOLR-5940: post.jar reports back detailed error in case of error responses.
(Sameer Maggon, shalin, Uwe Schindler)
- SOLR-6161: SolrDispatchFilter should throw java.lang.Error back even if wrapped in
another exception.
(Miklos Christine via shalin)
- SOLR-6153: ReplicationHandler backup response format should contain backup name.
(Varun Thacker via shalin)
- SOLR-6169: Remove broken handleAlias action in CoreAdminHandler
(Alan
Woodward)
- SOLR-6128: Removed deprecated analysis factories and fieldTypes from the example
schema.xml
(hossman)
- SOLR-5868: HttpClient should be configured to use ALLOW_ALL_HOSTNAME hostname
verifier to simplify SSL setup.
(Steve Davids via Mark Miller)
- Optimizations (1)
- SOLR-5681: Make the processing of Collection API calls multi-threaded.
(Anshum Gupta, shalin, Noble Paul)
- Build (2)
- SOLR-6006: Separate test and compile scope dependencies in the Solrj and
Solr contrib ivy.xml files, so that the derived Maven dependencies get
filled out properly in the corresponding POMs.
(Steven Scott, Steve Rowe)
- SOLR-6130: Added com.uwyn:jhighlight dependency to, and removed asm:asm
dependency from the extraction contrib - dependencies weren't fully
upgraded with the Tika 1.4->1.5 upgrade (SOLR-5763).
(Steve Rowe)
- Bug Fixes (10)
- SOLR-5904: ElectionContext can cancel an election when it should not if there
was an exception while trying to register as the leader.
(Mark Miller, Alan Woodward)
- SOLR-5993: ZkController can warn about shard leader conflict even after the conflict
is resolved.
(Gregory Chanan via shalin)
- SOLR-6017: Fix SimpleQParser to use query analyzer
(Ryan Ernst)
- SOLR-6029: CollapsingQParserPlugin throws ArrayIndexOutOfBoundsException
if elevated doc has been deleted from a segment.
(Greg Harris, Joel Bernstein)
- SOLR-6030: Use System.nanoTime() instead of currentTimeInMills() in LRUCache.warm.
(Tomás Fernández Löbbe via shalin)
- SOLR-6037: Fixed incorrect max/sum/stddev for Date fields in StatsComponent
(Brett Lucey, hossman)
- SOLR-6023: FieldAnalysisRequestHandler throws NPE if no parameters are supplied.
(shalin)
- SOLR-5090: SpellCheckComponent sometimes throws NPE if
"spellcheck.alternativeTermCount" is set to zero
(James Dyer).
- SOLR-6039: fixed debug output when no results in response
(Tomás Fernández Löbbe, hossman)
- SOLR-6035: CloudSolrServer directUpdate routing should use getCoreUrl.
(Marvin Justice, Joel Bernstein)
- Versions of Major Components (5)
- Apache Tika 1.5
- Carrot2 3.9.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.6
- Upgrading from Solr 4.7 (4)
- In previous versions of Solr, Terms that exceeded Lucene's MAX_TERM_LENGTH were
silently ignored when indexing documents. Beginning with Solr 4.8, a document
an error will be generated when attempting to index a document with a term
that is too large. If you wish to continue to have large terms ignored,
use "solr.LengthFilterFactory" in all of your Analyzers. See LUCENE-5472 for
more details.
- Solr 4.8 requires Java 7 or greater, Java 8 is verified to be
compatible and may bring some performance improvements. When using
Oracle Java 7 or OpenJDK 7, be sure to not use the GA build 147 or
update versions u40, u45 and u51! We recommend using u55 or later.
An overview of known JVM bugs can be found on
http://wiki.apache.org/lucene-java/JavaBugs
- ZooKeeper is upgraded from 3.4.5 to 3.4.6.
- <fields> and <types> tags have been deprecated. There is no longer any reason to
keep them in the schema file, they may be safely removed. This allows intermixing of
<fieldType>, <field> and <copyField> definitions if desired. Currently, these tags
are supported so either style may be implemented. TBD is whether they'll be
deprecated formally for 5.0
Detailed Change List
- System Requirements (1)
- LUCENE-4747, LUCENE-5514: Move to Java 7 as minimum Java version.
(Robert Muir, Uwe Schindler)
- New Features (20)
- SOLR-5130: Implement addReplica Collections API
(Noble Paul)
- SOLR-5183: JSON updates now support nested child documents using a
"_childDocument_" object key.
(Varun Thacker, hossman)
- SOLR-5714: You can now use one pool of memory for for the HDFS block cache
that all collections share.
(Mark Miller, Gregory Chanan)
- SOLR-5720: Add ExpandComponent to expand results collapsed by the
CollapsingQParserPlugin.
(Joel Bernstein)
- SOLR-3177: Enable tagging and excluding filters in StatsComponent via the
localParams syntax.
(Mathias H., Nikolai Luthman, Vitaliy Zhovtyuk, shalin)
- SOLR-1604: Wildcards, ORs etc inside Phrase Queries.
(Ahmet Arslan via Erick Erickson)
- SOLR-5477: Async execution of OverseerCollectionProcessor(CollectionsAPI)
tasks.
(Anshum Gupta)
- SOLR-5865: Provide a MiniSolrCloudCluster to enable easier testing.
(Greg Chanan via Mark Miller)
- SOLR-5860: Use leaderConflictResolveWait in WaitForState during recovery/startup,
improve logging and force refresh cluster state every 15 seconds.
(Timothy Potter via shalin)
- SOLR-5749: A new Overseer status collection API exposes overseer queue sizes, timing
statistics, success and error counts and last N failures per operation.
(shalin)
- SOLR-5858: Add a hl.qparser parameter to allow you to define a queryparser
for hl.q highlight queries. If no queryparser is defined, Solr will use
the overall query's defType.
(Alan Woodward)
- SOLR-4478: Allow cores to use configuration from a configsets directory
outside their instance directory.
(Alan Woodward, Erick Erickson)
- SOLR-5466: A new List collections and cluster status API which clients can use
to read collection and shard information instead of reading data directly from ZooKeeper.
(Dave Seltzer, Varun Thacker, Vitaliy Zhovtyuk, Erick Erickson, shalin)
- SOLR-5795: New DocExpirationUpdateProcessorFactory supports computing an expiration
date for documents from the "TTL" expression, as well as automatically deleting expired
documents on a periodic basis.
(hossman)
- SOLR-5829: Allow ExpandComponent to accept query and filter query parameters
(Joel Bernstein)
- SOLR-5653: Create a RestManager to provide REST API endpoints for
reconfigurable plugins.
(Tim Potter, Steve Rowe)
- SOLR-5655: Create a stopword filter factory that is (re)configurable,
and capable of reporting its configuration, via REST API.
(Tim Potter via Steve Rowe)
- SOLR-5654: Create a synonym filter factory that is (re)configurable, and
capable of reporting its configuration, via REST API.
(Tim Potter via Steve Rowe)
- SOLR-5960: Add support for basic authentication in post.jar tool, e.g.:
java -Durl="http://username:password@hostname:8983/solr/update" -jar post.jar sample.xml
(Sameer Maggon via Uwe Schindler)
- SOLR-4864: RegexReplaceProcessorFactory should support pattern capture group
substitution in replacement string.
(Sunil Srinivasan, Jack Krupansky via Steve Rowe)
- Bug Fixes (6)
- SOLR-5858, SOLR-4812: edismax and dismax query parsers can be used for parsing
highlight queries.
(Alan Woodward, Tien Nguyen Manh)
- SOLR-5893: On restarting overseer designate , move itself to front of the queue
(Noble Paul)
- SOLR-5915: Attempts to specify the parserImpl for
solr.PreAnalyzedField fieldtype failed.
(Mike McCandless)
- SOLR-5943: SolrCmdDistributor does not distribute the openSearcher parameter.
(ludovic Boutros via shalin)
- SOLR-5954: Slower DataImportHandler process caused by not reusing jdbc
connections.
(Mark Miller, Paco Garcia, Raja Nagendra Kumar)
- SOLR-5897: Upgraded to jQuery 1.7.2, Solr was previously using 1.4.3, the file was
mistakenly named 1.7.2
(steffkes)
- Optimizations (3)
- SOLR-1880: Distributed Search skips GET_FIELDS stage if EXECUTE_QUERY
stage gets all fields. Requests with fl=id or fl=id,score are now single-pass.
(Shawn Smith, Vitaliy Zhovtyuk, shalin)
- SOLR-5783: Requests to open a new searcher will now reuse the current registered
searcher (w/o additional warming) if possible in situations where the underlying
index has not changed. This reduces overhead in situations such as deletes that
do not modify the index, and/or redundant commits.
(hossman)
- SOLR-5884: When recovery is cancelled, any call to the leader to wait to see
the replica in the right state for recovery should be aborted.
(Mark Miller)
- Other Changes (23)
- SOLR-5909: Upgrade Carrot2 clustering dependency to 3.9.0.
(Dawid Weiss)
- SOLR-5764: Fix recently added tests to not use absolute paths to load test-files,
use SolrTestCaseJ4.getFile() and getResource() instead; fix morphlines/map-reduce
to not duplicate test resources and fix dependencies among them.
(Uwe Schindler)
- SOLR-5765: Update to SLF4J 1.7.6.
(Mark Miller)
- SOLR-5609: If legacy mode is disabled don't let cores create slices/replicas/collections .
All operations should be performed through collection API
(Noble Paul)
- SOLR-5613: Upgrade to commons-codec 1.9 for better BeiderMorseFilter performance.
(Thomas Champagne, Shawn Heisey via shalin)
- SOLR-5771: Add SolrTestCaseJ4.SuppressSSL annotation to disable SSL (instead of static boolean).
(Robert Muir)
- SOLR-5799: When registering as the leader, if an existing ephemeral
registration exists, wait a short time to see if it goes away.
(Mark Miller)
- LUCENE-5472: IndexWriter.addDocument will now throw an IllegalArgumentException
if a Term to be indexed exceeds IndexWriter.MAX_TERM_LENGTH. To recreate previous
behavior of silently ignoring these terms, use LengthFilter in your Analyzer.
(hossman, Mike McCandless, Varun Thacker)
- SOLR-5825: Separate http request creating and execution in SolrJ
(Steven Bower via Erick Erickson)
- SOLR-5837: Add hashCode/equals to SolrDocument, SolrInputDocument
and SolrInputField for testing purposes.
(Varun Thacker, Noble Paul,
Mark Miller)
- SOLR-5853: The createCollection methods in the test framework now reports
result of operation in the returned CollectionAdminResponse
(janhoy)
- SOLR-5838: Relative SolrHome Path Bug At AbstractFullDistribZkTestBase.
(Furkan KAMACI via shalin)
- SOLR-5763: Upgrade to Tika 1.5
(Vitaliy Zhovtyuk via Steve Rowe)
- SOLR-5881: Upgrade ZooKeeper to 3.4.6
(Shawn Heisey)
- SOLR-5883: Many tests do not shutdown SolrServer.
(Tomás Fernández Löbbe via Mark Miller)
- SOLR-5898: Update to latest Kite Morphlines release: Version 0.12.1.
(Mark Miller)
- SOLR-5228: Don't require <field> or <dynamicField> be inside of <fields> -- or
that <fieldType> be inside of <types>.
(Erick Erickson)
- SOLR-5903: SolrCore implements Closeable, cut over to using try-with-resources
where possible.
(Alan Woodward)
- SOLR-5914: Cleanup and fix Solr's test cleanup code.
(Mark Miller, Uwe Schindler)
- SOLR-5936: Deprecate non-Trie-based numeric & date field types.
(Steve Rowe)
- SOLR-5934: LBHttpSolrServer exception handling improvement and small test
improvements.
(Gregory Chanan via Mark Miller)
- SOLR-5773: CollapsingQParserPlugin should make elevated documents the
group head.
(David Boychuck, Joel Bernstein)
- SOLR-5937: Modernize the DIH example config sets.
(Steve Rowe)
- Versions of Major Components (5)
- Apache Tika 1.4
- Carrot2 3.8.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.5
Detailed Change List
- Bug Fixes (2)
- SOLR-5951: Fixed SolrDispatchFilter to throw useful exception on startup if
SLF4j logging jars are missing.
(Uwe Schindler, Hossman, Shawn Heisey)
- SOLR-5950: Maven config: make the org.slf4j:slf4j-api dependency transitive
(i.e., not optional) in all modules in which it's a dependency, including
solrj, except for the WAR, where it will remain optional.
(Uwe Schindler, Steve Rowe)
- Versions of Major Components (5)
- Apache Tika 1.4
- Carrot2 3.8.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.5
Detailed Change List
- Bug Fixes (28)
- SOLR-5647: The lib paths in example-schemaless will now load correctly.
(Paul Westin via Shawn Heisey)
- SOLR-5770: All attempts to match a SolrCore with its state in clusterstate.json
should be done with the CoreNodeName.
(Steve Davids via Mark Miller)
- SOLR-5875: QueryComponent.mergeIds() unmarshals all docs' sort field values once
per doc instead of once per shard.
(Alexey Serba, hoss, Martin de Vries via Steve Rowe)
- SOLR-5800: Admin UI - Analysis form doesn't render results correctly when a
CharFilter is used.
(steffkes)
- SOLR-5870: Admin UI - Reload on Core Admin doesn't show errors
(steffkes)
- SOLR-5867: OverseerCollectionProcessor isn't properly generating https urls in some
cases.
(Steve Davids via shalin)
- SOLR-5866: UpdateShardHandler needs to use the system default scheme registry to
properly handle https via javax.net.ssl.* properties.
(Steve Davids via shalin)
- SOLR-5782: The full MapReduceIndexer help text does not display when using --help.
(Mark Miller, Wolfgang Hoschek)
- SOLR-5824: Merge up Solr MapReduce contrib code to latest external changes.
Includes a few minor bug fixes.
(Mark Miller)
- SOLR-5818: distrib search with custom comparator does not quite work correctly
(Ryan Ernst)
- SOLR-5895: JavaBinLoader hides IOExceptions.
(Mike Sokolov via shalin)
- SOLR-5861: Recovery should not set onlyIfLeaderActive=true for slice in 'recovery'
state.
(shalin)
- SOLR-5423: CSV output doesn't include function field
(Arun Kumar, hossman, Steve Rowe)
- SOLR-5550: shards.info is not returned by a short circuited distributed query.
(Timothy Potter, shalin)
- SOLR-5777: Fix ordering of field values in JSON updates where
field name key is repeated
(hossman)
- SOLR-5734: We should use System.nanoTime rather than System.currentTimeMillis
when calculating elapsed time.
(Mark Miller, Ramkumar Aiyengar)
- SOLR-5760: ConcurrentUpdateSolrServer has a blockUntilFinished call when
streamDeletes is true that should be tucked into the if statement below it.
(Mark Miller, Gregory Chanan)
- SOLR-5761: HttpSolrServer has a few fields that can be set via setters but
are not volatile.
(Mark Miller, Gregory Chanan)
- SOLR-5907: The hdfs write cache can cause a reader to see a corrupted state.
It now defaults to off, and if you were using solr.hdfs.blockcache.write.enabled
explicitly, you should set it to false.
(Mark Miller)
- SOLR-5811: The Overseer will retry work items until success, which is a serious
problem if you hit a bad work item.
(Mark Miller)
- SOLR-5796: Increase how long we are willing to wait for a core to see the ZK
advertised leader in its local state.
(Timothy Potter, Mark Miller)
- SOLR-5834: Overseer threads are only being interrupted and not closed.
(hossman, Mark Miller)
- SOLR-5839: ZookeeperInfoServlet does not trim path properly.
(Furkan KAMACI via Mark Miller)
- SOLR-5874: Unsafe cast in CloudSolrServer's RouteException. Change
RouteException to handle Throwable rather than Exception.
(Mark Miller, David Arthur)
- SOLR-5899: CloudSolrServer's RouteResponse and RouteException should be
publicly accessible.
(Mark Miller, shalin)
- SOLR-5905: CollapsingQParserPlugin throws a NPE if required 'field' param is missing.
(Spyros Kapnissis via shalin)
- SOLR-5906: Collection create API ignores property.instanceDir parameter.
(Varun Thacker, shalin)
- SOLR-5920: Distributed sort on DateField, BoolField and BCD{Int,Long,Str}Field
returns string cast exception
(Eric Bus, AJ Lemke, hossman, Steve Rowe)
- Other Changes (1)
- SOLR-5796: Make how long we are willing to wait for a core to see the ZK
advertised leader in its local state configurable.
(Timothy Potter via Mark Miller)
- Versions of Major Components (5)
- Apache Tika 1.4
- Carrot2 3.8.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.5
- Upgrading from Solr 4.6.0 (2)
- CloudSolrServer and LBHttpSolrServer no longer declare MalformedURLException
as thrown from their constructors.
- Due to a bug in previous versions the default value of the 'discountOverlap' property
of DefaultSimilarity was not being set appropriately if you were using the implicit
DefaultSimilarityFactory instead of explicitly configuring it. To preserve
consistent behavior for people who upgrade, the implicit behavior is now contingent
on the <luceneMatchVersion/> -- discountOverlap=false for 4.6 and below,
discountOverlap=true for 4.7 and above. See SOLR-5561 for more information.
Detailed Change List
- New Features (32)
- SOLR-5308: SOLR-5601: SOLR-5710: A new 'migrate' collection API to split all
documents with a route key into another collection
(shalin)
- SOLR-5441: Expose number of transaction log files and their size via JMX.
(Rafał Kuć via shalin)
- SOLR-5320: Added support for tri-level compositeId routing.
(Anshum Gupta via shalin)
- SOLR-5458: Admin UI - Added a new "Files" conf directory browser/file viewer.
(steffkes)
- SOLR-5447, SOLR-5490: Add a QParserPlugin for Lucene's SimpleQueryParser.
(Jack Conradson via shalin)
- SOLR-5208: Support for the setting of core.properties key/values at create-time on
Collections API
(Erick Erickson)
- SOLR-5428: SOLR-5690: New 'stats.calcdistinct' parameter in StatsComponent returns
set of distinct values and their count. This can also be specified per field
e.g. 'f.field.stats.calcdistinct'.
(Elran Dvir via shalin)
- SOLR-5378, SOLR-5528: A new SuggestComponent that fully utilizes the Lucene suggester
module and adds pluggable dictionaries, payloads and better distributed support.
This is intended to eventually replace the Suggester support through the
SpellCheckComponent.
(Areek Zillur, Varun Thacker via shalin)
- SOLR-5492: Return the replica that actually served the query in shards.info
response.
(shalin)
- SOLR-5506: Support docValues in CollationField and ICUCollationField.
(Robert Muir)
- SOLR-5023: Add support for deleteInstanceDir to be passed from SolrJ for Core
Unload action.
(Lyubov Romanchuk, shalin)
- SOLR-1871: The 'map' function query accepts a ValueSource as target and
default value.
(Chris Harris, shalin)
- SOLR-5556: Allow class of CollectionsHandler and InfoHandler to be specified
in solr.xml.
(Gregory Chanan, Alan Woodward)
- SOLR-5581: Give ZkCLI the ability to get files.
(Gregory Chanan via Mark Miller)
- SOLR-5536: Add ValueSource collapse criteria to CollapsingQParsingPlugin
(Joel Bernstein)
- SOLR-5541: Allow QueryElevationComponent to accept elevateIds and excludeIds
as http parameters
(Joel Bernstein)
- SOLR-5463: new 'cursorMark' request param for deep paging of sorted result sets
(sarowe, hossman)
- SOLR-5529: Add support for queries to use multiple suggesters.
(Areek Zillur, Erick Erickson, via Robert Muir)
- SOLR-1301: Add a Solr contrib that allows for building Solr indexes via
Hadoop's MapReduce.
(Matt Revelle, Alexander Kanarsky, Steve Rowe,
Mark Miller, Greg Bowyer, Jason Rutherglen, Kris Jirapinyo, Jason Venner ,
Andrzej Bialecki, Patrick Hunt, Wolfgang Hoschek, Roman Shaposhnik,
Eric Wong)
- SOLR-5631: Add support for Lucene's FreeTextSuggester.
(Areek Zillur via Robert Muir)
- SOLR-5695: Add support for Lucene's BlendedInfixSuggester.
(Areek Zillur)
- SOLR-5476: Overseer Role for nodes
(Noble Paul)
- SOLR-5594: Allow FieldTypes to specify custom PrefixQuery behavior
(Anshum Gupta via hossman)
- LUCENE-5395: Upgrade to Spatial4j 0.4. Various new options are now exposed
automatically for an RPT field type. See Spatial4j CHANGES & javadocs.
https://github.com/spatial4j/spatial4j/blob/master/CHANGES.md
(David Smiley)
- SOLR-5670: allow _version_ to use DocValues.
(Per Steffensen via yonik)
- SOLR-5535: Set "partialResults" header for shards that error out if
shards.tolerant is specified.
(Steve Davids via shalin)
- SOLR-5610: Support cluster-wide properties with an API called CLUSTERPROP
(Noble Paul)
- SOLR-5623: Better diagnosis of RuntimeExceptions in analysis
(Benson Margulies)
- SOLR-5530: Added a NoOpResponseParser for SolrJ which puts the entire raw
response into an entry in the NamedList.
(Upayavira, Vitaliy Zhovtyuk via shalin)
- SOLR-5682: Make the admin InfoHandler more pluggable / derivable.
(Greg Chanan via Mark Miller)
- SOLR-5672: Add logParamsList parameter to support reduced logging.
(Christine Poerschke via Mark Miller)
- SOLR-3854: SSL support for SolrCloud.
(Sami Siren, hossman, Steve Davids,
Alexey Serba, Mark Miller)
- Bug Fixes (52)
- SOLR-5438: DebugComponent throws NPE when used with grouping.
(Tomás Fernández Löbbe via shalin)
- SOLR-4612: Admin UI - Analysis Screen contains empty table-columns
(steffkes)
- SOLR-5451: SyncStrategy closes its http connection manager before the
executor that uses it in its close method.
(Mark Miller)
- SOLR-5460: SolrDispatchFilter#sendError can get a SolrCore that it does not
close.
(Mark Miller)
- SOLR-5461: Request proxying should only set con.setDoOutput(true) if the
request is a post.
(Mark Miller)
- SOLR-5481: SolrCmdDistributor should not let the http client do its own
retries.
(Mark Miller)
- LUCENE-5347: Fixed Solr's Zookeeper Client to copy files to Zookeeper using
binary transfer. Previously data was read with default encoding and stored
in zookeeper as UTF-8. This bug was found after upgrading to forbidden-apis
1.4.
(Uwe Schindler)
- SOLR-4376: DataImportHandler uses wrong date format for last_index_time if
a delta-import is run first before any full-imports.
(Sebastien Lorber, Arcadius Ahouansou via shalin)
- SOLR-5494: CoreContainer#remove throws NPE rather than returning null when
a SolrCore does not exist in core discovery mode.
(Mark Miller)
- SOLR-5354: Distributed sort is broken with CUSTOM FieldType.
(Steve Rowe, hossman, Robert Muir, Jessica Cheng)
- SOLR-5515: NPE when getting stats on date field with empty result on
SolrCloud.
(Alexander Sagen, shalin)
- SOLR-5204: StatsComponent and SpellCheckComponent do not support the
shards.tolerant=true parameter.
(Anca Kopetz, shalin)
- SOLR-5527: DIH logs spurious warning for special commands.
(shalin)
- SOLR-5524: Exception when using Query Function inside Scale Function.
(Trey Grainger, yonik)
- SOLR-5562: ConcurrentUpdateSolrServer constructor ignores supplied httpclient.
(Kyle Halliday via Mark Miller)
- SOLR-5567: ZkController getHostAddress duplicates url prefix.
(Kyle Halliday, Alexey Serba, shalin)
- SOLR-4992: Solr eats OutOfMemoryError exceptions in many cases.
(Mark Miller, Daniel Collins)
- LUCENE-5399, SOLR-5354 sort wouldn't work correctly with
distributed searching for some field types such as legacy numeric
types
(Rob Muir, Mike McCandless)
- SOLR-5643: ConcurrentUpdateSolrServer will sometimes not spawn a new Runner
thread even though there are updates in the queue.
(Mark Miller)
- SOLR-5650: When a replica becomes a leader, only peer sync with other replicas
that last published an ACTIVE state.
(Mark Miller)
- SOLR-5657: When a SolrCore starts on HDFS, it should gracefully handle HDFS
being in safe mode.
(Mark Miller)
- SOLR-5663: example-DIH uses non-existing column for mapping (case-sensitive)
(steffkes)
- SOLR-5666: Using the hdfs write cache can result in appearance of corrupted
index.
(Mark Miller)
- SOLR-5230: Call DelegatingCollector.finish() during grouping.
(Joel Bernstein, ehatcher)
- SOLR-5679: Shard splitting fails with ClassCastException on collections
upgraded from 4.5 and earlier versions.
(Brett Hoerner, shalin)
- SOLR-5673: HTTPSolrServer doesn't set own property correctly in
setFollowRedirects.
(Frank Wesemann via shalin)
- SOLR-5676: SolrCloud updates rejected if talking to secure ZooKeeper.
(Greg Chanan via Mark Miller)
- SOLR-5634: SolrJ GroupCommand.getNGroups returns null if group.format=simple
and group.ngroups=true.
(Artem Lukanin via shalin)
- SOLR-5667: Performance problem when not using hdfs block cache.
(Mark Miller)
- SOLR-5526: Fixed NPE that could arise when explicitly configuring some built
in QParserPlugins
(Nikolay Khitrin, Vitaliy Zhovtyuk, hossman)
- SOLR-5598: LanguageIdentifierUpdateProcessor ignores all but the first value
of multiValued string fields.
(Andreas Hubold, Vitaliy Zhovtyuk via shalin)
- SOLR-5593: Replicas should accept the last updates from a leader that has just
lost its connection to ZooKeeper.
(Christine Poerschke via Mark Miller)
- SOLR-5678: SolrZkClient should throw a SolrException when connect times out
rather than a RuntimeException.
(Karl Wright, Anshum Gupta, Mark Miller)
- SOLR-4072: Error message is incorrect for linkconfig in ZkCLI.
(Vamsee Yarlagadda, Adam Hahn, via Mark Miller)
- SOLR-5691: Sharing non thread safe WeakHashMap across thread can cause
problems.
(Bojan Smid, Mark Miller)
- SOLR-5693: Running on HDFS does work correctly with NRT search.
(Mark Miller)
- SOLR-5644: SplitShard does not handle not finding a shard leader well.
(Mark Miller, Anshum Gupta via shalin)
- SOLR-5704: coreRootDirectory was not respected when creating new cores
via CoreAdminHandler
(Jesse Sipprell, Alan Woodward)
- SOLR-5709: Highlighting grouped duplicate docs from different shards with
group.limit > 1 throws ArrayIndexOutOfBoundsException.
(Steve Rowe)
- SOLR-5561: Fix implicit DefaultSimilarityFactory initialization in IndexSchema
to properly specify discountOverlap option.
(Isaac Hebsh, Ahmet Arslan, Vitaliy Zhovtyuk, hossman)
- SOLR-5689: On reconnect, ZkController cancels election on first context rather
than latest.
(Gregory Chanan, Mark Miller via shalin)
- SOLR-5649: Clean up some minor ConnectionManager issues.
(Mark Miller, Gregory Chanan)
- SOLR-5365: Fix bug with compressed files in ExtractingRequestHandler by
upgrading commons-compress to 1.7
(Jan Høydahl, hossman)
- SOLR-5675: cloud-scripts/zkcli.bat: quote option log4j
(Günther Ruck via steffkes
- SOLR-5721: ConnectionManager can become stuck in likeExpired.
(Gregory Chanan via Mark Miller)
- SOLR-5731: In ConnectionManager, we should catch and only log exceptions
from BeforeReconnect.
(Mark Miller)
- SOLR-5718: Make LBHttpSolrServer zombie checks non-distrib and non-scoring.
(Christine Poerschke via Mark Miller)
- SOLR-5727: LBHttpSolrServer should only retry on Connection exceptions when
sending updates. Affects CloudSolrServer.
(Mark Miller)
- SOLR-5739: Sub-shards created by shard splitting have their update log set
to buffering mode on restarts.
(Günther Ruck, shalin)
- SOLR-5741: UpdateShardHandler was not correctly setting max total connections
on the HttpClient.
(Shawn Heisey)
- SOLR-5620: ZKStateReader.aliases should be volatile to ensure all threads see
the latest aliases.
(Ramkumar Aiyengar via Mark Miller)
- SOLR-5448: ShowFileRequestHandler treats everything as Directory, when in
Cloud-Mode.
(Erick Erickson, steffkes)
- Optimizations (8)
- SOLR-5436: Eliminate the 1500ms wait in overseer loop as well as
polling the ZK distributed queue.
(Noble Paul, Mark Miller)
- SOLR-5189: Solr 4.x Web UI Log Viewer does not display 'date' column from
logs
(steffkes)
- SOLR-5512: Optimize DocValuesFacets.
(Robert Muir)
- SOLR-2960: fix DIH XPathEntityProcessor to add the correct number of "null"
placeholders for multi-valued fields
(Michael Watts via James Dyer)
- SOLR-5214: Reduce memory usage for shard splitting by merging segments one
at a time.
(Christine Poerschke via shalin)
- SOLR-4227: Wrap XML RequestWriter's OutputStreamWriter in a BufferedWriter
to avoid frequent converter invocations.
(Conrad Herrmann, shalin)
- SOLR-5624: Enable QueryResultCache for CollapsingQParserPlugin.
(David Boychuck, Joel Bernstein)
- LUCENE-5440: DocSet decoupled from OpenBitSet. DocSetBase moved to use
FixedBitSet instead of OpenBitSet. As a result BitDocSet now only works
with FixedBitSet.
(Shai Erera)
- Other Changes (23)
- SOLR-5399: Add distributed request tracking information to DebugComponent
(Tomás Fernández Löbbe via Ryan Ernst)
- SOLR-5421: Remove double set of distrib.from param in processAdd method of
DistributedUpdateProcessor.
(Anshum Gupta via shalin)
- SOLR-5404: The example config references deprecated classes.
(Uwe Schindler, Rafał Kuć via Mark Miller)
- SOLR-5487: Replication factor error message doesn't match constraint.
(Patrick Hunt via shalin)
- SOLR-5499: Log a warning if /get is not registered when using SolrCloud.
(Daniel Collins via shalin)
- SOLR-5517: Return HTTP error on POST requests with no Content-Type.
(Ryan Ernst, Uwe Schindler)
- SOLR-5502: Added a test for tri-level compositeId routing with documents
having a "/" in a document id.
(Anshum Gupta via Mark Miller)
- SOLR-5533: Improve out of the box support for running Solr on hdfs with
SolrCloud.
(Mark Miller)
- SOLR-5548: Give DistributedSearchTestCase / JettySolrRunner the ability to
specify extra filters.
(Greg Chanan via Mark Miller)
- SOLR-5555: LBHttpSolrServer and CloudSolrServer constructors don't need to
declare MalformedURLExceptions
(Sushil Bajracharya, Alan Woodward)
- SOLR-5565: Raise default ZooKeeper session timeout to 30 seconds from 15
seconds.
(Mark Miller)
- SOLR-5574: CoreContainer shutdown publishes all nodes as down and waits to
see that and then again publishes all nodes as down.
(Mark Miller)
- SOLR-5590: Upgrade HttpClient/HttpComponents to 4.3.x.
(Karl Wright via Shawn Heisey)
- SOLR-2794: change the default of hl.phraseLimit to 5000.
(Michael Della Bitta via Robert Muir, Koji, zarni - pull request #11)
- SOLR-5632: Improve response message for reloading a non-existent core.
(Anshum Gupta via Mark Miller)
- SOLR-5633: HttpShardHandlerFactory should make its http client available to subclasses.
(Ryan Ernst)
- SOLR-5684: Shutdown SolrServer clients created in BasicDistributedZk2Test and
BasicDistributedZkTest.
(Tomás Fernández Löbbe via shalin)
- SOLR-5629: SolrIndexSearcher.name should include core name.
(Shikhar Bhushan via shalin)
- SOLR-5702: Log config name found for collection at info level.
(Christine Poerschke via Mark Miller)
- SOLR-5659: Add test for compositeId ending with an '!'.
(Markus Jelsma, Anshum Gupta via shalin)
- SOLR-5700: Improve error handling of remote queries (proxied requests).
(Greg Chanan, Steve Davids via Mark Miller)
- SOLR-5585: Raise Collections API timeout to 3 minutes from one minute.
(Mark Miller)
- SOLR-5257: Improved error/warn messages when Update XML contains unexpected XML nodes
(Vitaliy Zhovtyuk, hossman)
- Versions of Major Components (5)
- Apache Tika 1.4
- Carrot2 3.8.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.5
Detailed Change List
- Bug Fixes (29)
- SOLR-5408: CollapsingQParserPlugin scores incorrectly when multiple sort criteria are used
(Brandon Chapman, Joel Bernstein)
- SOLR-5416: CollapsingQParserPlugin breaks Tag/Exclude Faceting
(David Boychuck, Joel Bernstein)
- SOLR-5442: Python client cannot parse proxied response when served by Tomcat.
(Patrick Hunt, Gregory Chanan, Vamsee Yarlagadda, Romain Rigaux, Mark Miller)
- SOLR-5445: Proxied responses should propagate all headers rather than the
first one for each key.
(Patrick Hunt, Mark Miller)
- SOLR-5479: SolrCmdDistributor retry logic stops if a leader for the request
cannot be found in 1 second.
(Mark Miller)
- SOLR-5532: SolrJ Content-Type validation is too strict for some
webcontainers / proxies.
(Jakob Furrer, hossman, Shawn Heisey, Uwe Schindler,
Mark Miller)
- SOLR-5547: Creating a collection alias using SolrJ's CollectionAdminRequest
sets the alias name and the collections to alias to the same value.
(Aaron Schram, Mark Miller)
- SOLR-5577: Likely ZooKeeper expiration should not slow down updates a given
amount, but instead cut off updates after a given time.
(Mark Miller, Christine Poerschke, Ramkumar Aiyengar)
- SOLR-5580: NPE when creating a core with both explicit shard and coreNodeName.
(YouPeng Yang, Mark Miller)
- SOLR-5552: Leader recovery process can select the wrong leader if all replicas
for a shard are down and trying to recover as well as lose updates that should
have been recovered.
(Timothy Potter, Mark Miller)
- SOLR-5569 A replica should not try and recover from a leader until it has
published that it is ACTIVE.
(Mark Miller)
- SOLR-5568 A SolrCore cannot decide to be the leader just because the cluster
state says no other SolrCore's are active.
(Mark Miller)
- SOLR-5496: We should share an http connection manager across non search
HttpClients and ensure all http connection managers get shutdown.
(Mark Miller)
- SOLR-5583: ConcurrentUpdateSolrServer#blockUntilFinished may wait forever if
the executor service is shutdown.
(Mark Miller)
- SOLR-5586: All ZkCmdExecutor's should be initialized with the zk client
timeout.
(Mark Miller)
- SOLR-5587: ElectionContext implementations should use
ZkCmdExecutor#ensureExists to ensure their election paths are properly
created.
(Mark Miller)
- SOLR-5540: HdfsLockFactory should explicitly create the lock parent directory if
necessary.
(Mark Miller)
- SOLR-4709: The core reload after replication if config files have changed
can fail due to a race condition.
(Mark Miller, Hossman)
- SOLR-5503: Retry 'forward to leader' requests less aggressively - rather
than on IOException and status 500, ConnectException.
(Mark Miller)
- SOLR-5588: PeerSync doesn't count all connect failures as success.
(Mark Miller)
- SOLR-5564: hl.maxAlternateFieldLength should apply to original field when
fallback is attempted
(janhoy)
- SOLR-5608: Don't allow a closed SolrCore to publish state to ZooKeeper.
(Mark Miller, Shawn Heisey)
- SOLR-5615: Deadlock while trying to recover after a ZK session expiration.
(Ramkumar Aiyengar, Mark Miller)
- SOLR-5543: Core swaps resulted in duplicate core entries in solr.xml when
using solr.xml persistence.
(Bill Bell, Alan Woodward)
- SOLR-5618: Fix false cache hits in queryResultCache when hashCodes are equal
and duplicate filter queries exist in one of the requests
(hossman)
- SOLR-4260: ConcurrentUpdateSolrServer#blockUntilFinished can return before
all previously added updates have finished. This could cause distributed
updates meant for replicas to be lost.
(Markus Jelsma, Timothy Potter,
Joel Bernstein, Mark Miller)
- SOLR-5645: A SolrCore reload via the CoreContainer will try and register in
zk again with the new SolrCore.
(Mark Miller)
- SOLR-5636: SolrRequestParsers does some xpath lookups on every request, which
can cause concurrency issues.
(Mark Miller)
- SOLR-5658: commitWithin and overwrite are not being distributed to replicas
now that SolrCloud uses javabin to distribute updates.
(Mark Miller, Varun Thacker, Elodie Sannier, shalin)
- Optimizations (1)
- SOLR-5576: Improve concurrency when registering and waiting for all
SolrCore's to register a DOWN state.
(Christine Poerschke via Mark Miller)
- Versions of Major Components (5)
- Apache Tika 1.4
- Carrot2 3.8.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.5
- Upgrading from Solr 4.5.0 (1)
- If you are using methods from FieldMutatingUpdateProcessorFactory for getting
configuration information (oneOrMany or getBooleanArg), those methods have
been moved to NamedList and renamed to removeConfigArgs and removeBooleanArg,
respectively. The original methods are deprecated, to be removed in 5.0.
See SOLR-5264.
Detailed Change List
- New Features (17)
- SOLR-5167: Add support for AnalyzingInfixSuggester (AnalyzingInfixLookupFactory).
(Areek Zillur, Varun Thacker via Robert Muir)
- SOLR-5246: Shard splitting now supports collections configured with router.field.
(shalin)
- SOLR-5274: Allow JettySolrRunner SSL config to be specified via a constructor.
(Mark Miller)
- SOLR-5300: Shards can be split by specifying arbitrary number of hash ranges
within the shard's hash range.
(shalin)
- SOLR-5226: Add Lucene index heap usage to the Solr admin UI.
(Areek Zillur via Robert Muir)
- SOLR-5324: Make sub shard replica recovery and shard state switch asynchronous.
(Yago Riveiro, shalin)
- SOLR-5338: Split shards by a route key using split.key parameter.
(shalin)
- SOLR-5353: Enhance CoreAdmin api to split a route key's documents from an index
and leave behind all other documents.
(shalin)
- SOLR-5027: CollapsingQParserPlugin for high performance field collapsing on high cardinality fields.
(Joel Bernstein)
- SOLR-5395: Added a RunAlways marker interface for UpdateRequestProcessorFactory
implementations indicating that they should not be removed in later stages
of distributed updates (usually signalled by the update.distrib parameter)
(yonik)
- SOLR-5310: Add a collection admin command to remove a replica
(noble)
- SOLR-5311: Avoid registering replicas which are removed
(noble)
- SOLR-5406: CloudSolrServer failed to propagate request parameters
along with delete updates.
(yonik)
- SOLR-5374: Support user configured doc-centric versioning rules
via the optional DocBasedVersionConstraintsProcessorFactory
update processor
(Hossman, yonik)
- SOLR-5392: Extend solrj apis to cover collection management.
(Roman Shaposhnik via Mark Miller)
- SOLR-5084: new field type EnumField.
(Elran Dvir via Erick Erickson)
- SOLR-5464: Add option to ConcurrentSolrServer to stream pure delete
requests.
(Mark Miller)
- Bug Fixes (14)
- SOLR-5216: Document updates to SolrCloud can cause a distributed deadlock.
(Mark Miller)
- SOLR-5367: Unmarshalling delete by id commands with JavaBin can lead to class cast
exception.
(Mark Miller)
- SOLR-5359: ZooKeeper client is not closed when it fails to connect to an ensemble.
(Mark Miller, Klaus Herrmann)
- SOLR-5042: MoreLikeThisComponent was using the rows/count value in place of
flags, which caused a number of very strange issues, including NPEs and
ignoring requests for the results to include the score.
(Anshum Gupta, Mark Miller, Shawn Heisey)
- SOLR-5371: Solr should consistently call SolrServer#shutdown
(Mark Miller)
- SOLR-5363: Solr doesn't start up properly with Log4J2
(Petar Tahchiev via Alan
Woodward)
- SOLR-5380: Using cloudSolrServer.setDefaultCollection(collectionId) does not
work as intended for an alias spanning more than 1 collection.
(Thomas Egense, Shawn Heisey, Mark Miller)
- SOLR-5418: Background merge after field removed from solr.xml causes error.
(Reported on user's list, Robert M's patch via Erick Erickson)
- SOLR-5318: Creating a core via the admin API doesn't respect transient property
(Olivier Soyez via Erick Erickson)
- SOLR-5388: Creating a new core via the HTTP API that results in a transient being
unloaded results in a " Too many close [count:-1]" error.
(Olivier Soyez via Erick Erickson)
- SOLR-5453: Raise recovery socket read timeouts.
(Mark Miller)
- SOLR-5397: Replication can fail silently in some cases.
(Mark Miller)
- SOLR-5465: SolrCmdDistributor retry logic has a concurrency race bug.
(Mark Miller)
- SOLR-5452: Do not attempt to proxy internal update requests.
(Mark Miller)
- Optimizations (4)
- SOLR-5232: SolrCloud should distribute updates via streaming rather than buffering.
(Mark Miller)
- SOLR-5223: SolrCloud should use the JavaBin binary format for communication by default.
(Mark Miller)
- SOLR-5370: Requests to recover when an update fails should be done in
background threads.
(Mark Miller)
- LUCENE-5300,LUCENE-5304: Specialized faceting for fields which are declared as
multi-valued in the schema but are actually single-valued.
(Adrien Grand)
- Security (1)
- SOLR-4882: SolrResourceLoader was restricted to only allow access to resource
files below the instance dir. The reason for this is security related: Some
Solr components allow to pass in resource paths via REST parameters
(e.g. XSL stylesheets, velocity templates,...) and load them via resource
loader. For backwards compatibility, this security feature can be disabled
by a new system property: solr.allow.unsafe.resourceloading=true
(Uwe Schindler)
- Other Changes (8)
- SOLR-5237: Add indexHeapUsageBytes to LukeRequestHandler, indicating how much
heap memory is being used by the underlying Lucene index structures.
(Areek Zillur via Robert Muir)
- SOLR-5241: Fix SimplePostToolTest performance problem - implicit DNS lookups
(hossman)
- SOLR-5273: Update HttpComponents to 4.2.5 and 4.2.6.
(Mark Miller)
- SOLR-5264: Move methods for getting config information from
FieldMutatingUpdateProcessorFactory to NamedList.
(Shawn Heisey)
- SOLR-5319: Remove unused and incorrect router name from Collection ZK nodes.
(Jessica Cheng via shalin)
- SOLR-5321: Remove unnecessary code in Overseer.updateState method which tries to
use router name from message where none is ever sent.
(shalin)
- SOLR-5401: SolrResourceLoader logs a warning if a deprecated (factory) class
is used in schema or config.
(Uwe Schindler)
- SOLR-3397: Warn if master or slave replication is enabled in SolrCloud mode.
(Erick
Erickson)
- Versions of Major Components (5)
- Apache Tika 1.4
- Carrot2 3.8.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.5
Detailed Change List
- Bug Fixes (16)
- SOLR-4590: Collections API should return a nice error when not in SolrCloud mode.
(Anshum Gupta, Mark Miller)
- SOLR-5295: The CREATESHARD collection API creates maxShardsPerNode number of
replicas if replicationFactor is not specified.
(Brett Hoerner, shalin)
- SOLR-5296: Creating a collection with implicit router adds shard ranges
to each shard.
(shalin)
- SOLR-5263: Fix CloudSolrServer URL cache update race.
(Jessica Cheng, Mark Miller)
- SOLR-5297: Admin UI - Threads Screen missing Icon
(steffkes)
- SOLR-5301: DELETEALIAS command prints CREATEALIAS in logs
(janhoy)
- SOLR-5255: Remove unnecessary call to fetch and watch live nodes in ZkStateReader
cluster watcher.
(Jessica Cheng via shalin)
- SOLR-5305: Admin UI - Reloading System-Information on Dashboard does not work
anymore
(steffkes)
- SOLR-5314: Shard split action should use soft commits instead of hard commits
to make sub shard data visible.
(Kalle Aaltonen, shalin)
- SOLR-5327: SOLR-4915, "The root cause should be returned to the user when a SolrCore create
call fails", was reverted.
(Mark Miller)
- SOLR-5317: SolrCore persistence bugs if defining SolrCores in solr.xml.
(Mark Miller, Yago Riveiro)
- SOLR-5306: Extra collection creation parameters like collection.configName are
not being respected.
(Mark Miller, Liang Tianyu, Nathan Neulinger)
- SOLR-5325: ZooKeeper connection loss can cause the Overseer to stop processing
commands.
(Christine Poerschke, Mark Miller, Jessica Cheng)
- SOLR-4327: HttpSolrServer can leak connections on errors.
(Karl Wright, Mark Miller)
- SOLR-5349: CloudSolrServer - ZK timeout arguments passed to ZkStateReader are flipped.
(Ricardo Merizalde via shalin)
- SOLR-5330: facet.method=fcs on single values fields could sometimes result
in incorrect facet labels.
(Michael Froh, yonik)
- Other Changes (1)
- SOLR-5323: Disable ClusteringComponent by default in collection1 example.
The solr.clustering.enabled system property needs to be set to 'true'
to enable the clustering contrib (reverts SOLR-4708).
(Dawid Weiss)
- Versions of Major Components (5)
- Apache Tika 1.4
- Carrot2 3.8.0
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.5
- Upgrading from Solr 4.4.0 (5)
- XML configuration parsing is now more strict about situations where a single
setting is allowed but multiple values are found. In the past, one value
would be chosen arbitrarily and silently. Starting with 4.5, configuration
parsing will fail with an error in situations like this. If you see error
messages such as "solrconfig.xml contains more than one value for config path:
XXXXX" or "Found Z configuration sections when at most 1 is allowed matching
expression: XXXXX" check your solrconfig.xml file for multiple occurrences of
XXXXX and delete the ones that you do not wish to use. See SOLR-4953 &
SOLR-5108 for more details.
- In the past, schema.xml parsing would silently ignore "default" or "required"
options specified on <dynamicField/> declarations. Beginning with 4.5, attempting
to do configured these on a dynamic field will cause an init error. If you
encounter one of these errors when upgrading an existing schema.xml, you can
safely remove these attributes, regardless of their value, from your config and
Solr will continue to behave exactly as it did in previous versions. See
SOLR-5227 for more details.
- The UniqFieldsUpdateProcessorFactory has been improved to support all of the
FieldMutatingUpdateProcessorFactory selector options. The <lst named="fields">
init param option is now deprecated and should be replaced with the more standard
<arr name="fieldName">. See SOLR-4249 for more details.
- UpdateRequestExt has been removed as part of SOLR-4816. You should use UpdateRequest
instead.
- CloudSolrServer can now use multiple threads to add documents by default. This is a
small change in runtime semantics when using the bulk add method - you will still
end up with the same exception on a failure, but some documents beyond the one that
failed may have made it in. To get the old, single threaded behavior, set parallel updates
to false on the CloudSolrServer instance.
Detailed Change List
- New Features (18)
- SOLR-5219: Rewritten selection of the default search and document clustering
algorithms.
(Dawid Weiss)
- SOLR-5202: Support easier overrides of Carrot2 clustering attributes via
XML data sets exported from the Workbench.
(Dawid Weiss)
- SOLR-5126: Update Carrot2 clustering to version 3.8.0, update Morfologik
to version 1.7.1
(Dawid Weiss)
- SOLR-2345: Enhanced geodist() to work with an RPT field, provided that the
field is referenced via 'sfield' and the query point is constant.
(David Smiley)
- SOLR-5082: The encoding of URL-encoded query parameters can be changed with
the "ie" (input encoding) parameter, e.g. "select?q=m%FCller&ie=ISO-8859-1".
The default is UTF-8. To change the encoding of POSTed content, use the
"Content-Type" HTTP header.
(Uwe Schindler, Shawn Heisey)
- SOLR-4221: Custom sharding
(Noble Paul)
- SOLR-4808: Persist and use router,replicationFactor and maxShardsPerNode at Collection
and Shard level
(Noble Paul, Shalin Mangar)
- SOLR-5006: CREATESHARD command for 'implicit' shards
(Noble Paul)
- SOLR-5017: Allow sharding based on the value of a field
(Noble Paul)
- SOLR-4222: create custom sharded collection via collections API
(Noble Paul)
- SOLR-4718: Allow solr.xml to be stored in ZooKeeper.
(Mark Miller, Erick Erickson)
- SOLR-5156: Enhance ZkCLI to allow uploading of arbitrary files to ZK.
(Erick Erickson)
- SOLR-5165: Single-valued docValues fields no longer require a default value.
Additionally they work with sortMissingFirst, sortMissingLast, facet.missing,
exists() in function queries, etc.
(Robert Muir)
- SOLR-5182: Add NoOpRegenerator, a regenerator for custom per-segment caches
where items are preserved across commits.
(Robert Muir)
- SOLR-4249: UniqFieldsUpdateProcessorFactory now extends
FieldMutatingUpdateProcessorFactory and supports all of its selector options. Use
of the "fields" init param is now deprecated in favor of "fieldName"
(hossman)
- SOLR-2548: Allow multiple threads to be specified for faceting. When threading, one
can specify facet.threads to parallelize loading the uninverted fields. In at least
one extreme case this reduced warmup time from 20 seconds to 3 seconds.
(Janne Majaranta,
Gun Akkor via Erick Erickson, David Smiley)
- SOLR-4816: CloudSolrServer can now route updates locally and no longer relies on inter-node
update forwarding.
(Joel Bernstein, Shikhar Bhushan, Stephen Riesenberg, Mark Miller)
- SOLR-3249: Allow CloudSolrServer and SolrCmdDistributor to use JavaBin.
(Mark Miller)
- Bug Fixes (30)
- SOLR-3633: web UI reports an error if CoreAdminHandler says there are no
SolrCores
(steffkes)
- SOLR-4489: SpellCheckComponent can throw StringIndexOutOfBoundsException
when generating collations involving multiple word-break corrections.
(James Dyer)
- SOLR-5087 - CoreAdminHandler.handleMergeAction generating NullPointerException
(Patrick Hunt via Erick Erickson)
- SOLR-5107: Fixed NPE when using numTerms=0 in LukeRequestHandler
(Ahmet Arslan, hossman)
- SOLR-4679, SOLR-4908, SOLR-5124: Text extracted from HTML or PDF files
using Solr Cell was missing ignorable whitespace, which is inserted by
TIKA for convenience to support plain text extraction without using the
HTML elements. This bug resulted in glued words.
(hossman, Uwe Schindler)
- SOLR-5121: zkcli usage help for makepath doesn't match actual command.
(Daniel Collins via Mark Miller)
- SOLR-5119: Managed schema problems after adding fields via Schema Rest API.
(Nils Kübler, Steve Rowe)
- SOLR-5133: HdfsUpdateLog can fail to close a FileSystem instance if init
is called more than once.
(Mark Miller)
- SOLR-5135: Harden Collection API deletion of /collections/$collection
ZooKeeper node.
(Mark Miller)
- SOLR-4764: When using NRT, just init the first reader from IndexWriter.
(Robert Muir, Mark Miller)
- SOLR-5122: Fixed bug in spellcheck.collateMaxCollectDocs. Eliminates risk
of divide by zero, and makes estimated hit counts meaningful in non-optimized
indexes.
(hossman)
- SOLR-3936: Fixed QueryElevationComponent sorting when used with Grouping
(Michael Garski via hossman)
- SOLR-5171: SOLR Admin gui works in IE9, breaks in IE10.
(Joseph L Howard via
steffkes)
- SOLR-5174: Admin UI - Query View doesn't highlight (json) Result if it
contains HTML Tags
(steffkes)
- SOLR-4817 Solr should not fall back to the back compat built in solr.xml in SolrCloud
mode
(Erick Erickson)
- SOLR-5112: Show full message in Admin UI Logging View
(Matthew Keeney via
steffkes)
- SOLR-5190: SolrEntityProcessor substitutes variables only once in child entities
(Harsh Chawla, shalin)
- SOLR-3852: Fixed ZookeeperInfoServlet so that the SolrCloud Admin UI pages will
work even if ZK contains nodes with data which are not utf8 text.
(hossman)
- SOLR-5206: Fixed OpenExchangeRatesOrgProvider to use refreshInterval correctly
(Catalin, hossman)
- SOLR-5215: Fix possibility of deadlock in ZooKeeper ConnectionManager.
(Mark Miller, Ricardo Merizalde)
- SOLR-4909: Use DirectoryReader.openIfChanged in non-NRT mode.
(Michael Garski via Robert Muir)
- SOLR-5227: Correctly fail schema initialization if a dynamicField is configured to
be required, or have a default value.
(hossman)
- SOLR-5231: Fixed a bug with the behavior of BoolField that caused documents w/o
a value for the field to act as if the value were true in functions if no other
documents in the same index segment had a value of true.
(Robert Muir, hossman, yonik)
- SOLR-5233: The "deleteshard" collections API doesn't wait for cluster state to update,
can fail if some nodes of the deleted shard were down and had incorrect logging.
(Christine Poerschke, shalin)
- SOLR-5150: HdfsIndexInput may not fully read requested bytes.
(Mark Miller, Patrick Hunt)
- SOLR-5240: All solr cores will now be loaded in parallel (as opposed to a fixed number)
in zookeeper mode to avoid deadlocks due to replicas waiting for other replicas
to come up.
(yonik)
- SOLR-5243: Killing a shard in one collection can result in leader election in a different
collection if they share the same coreNodeName.
(yonik, Mark Miller)
- SOLR-5281: IndexSchema log message was printing '[null]' instead of
'[<core name>]'
(Jun Ohtani via Steve Rowe)
- SOLR-5279: Implicit properties don't seem to exist on core RELOAD
(elyograg, hossman, Steve Rowe)
- SOLR-5291: Solrj does not propagate the root cause to the user for many errors.
(Mark Miller)
- Optimizations (5)
- SOLR-5044: Admin UI - Note on Core-Admin about directories while creating
core
(steffkes)
- SOLR-5134: Have HdfsIndexOutput extend BufferedIndexOutput.
(Mark Miller, Uwe Schindler)
- SOLR-5057: QueryResultCache should not related with the order of fq's list
(Feihong Huang via Erick Erickson)
- SOLR-4816: CloudSolrServer now uses multiple threads to send updates by default.
(Joel Bernstein via Mark Miller)
- SOLR-3530: Better error messages / Content-Type validation in SolrJ.
(Mark Miller, hossman)
- Other Changes (7)
- SOLR-4708: Enable ClusteringComponent by default in collection1 example.
The solr.clustering.enabled system property is set to 'true' by default.
(ehatcher, Dawid Weiss)
- SOLR-4914, SOLR-5162: Factor out core list persistence and discovery into a
new CoresLocator interface.
(Alan Woodward, Shawn Heisey)
- SOLR-5056: Improve type safety of ConfigSolr class.
(Alan Woodward)
- SOLR-4951: Better randomization of MergePolicy in Solr tests
(hossman)
- SOLR-4953, SOLR-5108: Make XML Configuration parsing fail if an xpath matches
multiple nodes when only a single value or plugin instance is expected.
(hossman)
- The routing parameter "shard.keys" is deprecated as part of SOLR-5017 .The new parameter name is '_route_' .
The old parameter should continue to work for another release
(Noble Paul)
- SOLR-5173: Solr-core's Maven configuration includes test-only Hadoop
dependencies as indirect compile-time dependencies.
(Chris Collins, Steve Rowe)
- Versions of Major Components (5)
- Apache Tika 1.4
- Carrot2 3.6.2
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.5
- Upgrading from Solr 4.3.0 (3)
- TieredMergePolicy and the various subtypes of LogMergePolicy no longer have
an explicit "setUseCompoundFile" method. Instead the behavior of new
segments is determined by the IndexWriter configuration, and the MergePolicy
is only consulted to determine if merge segments should use the compound
file format (based on the value of "setNoCFSRatio"). If you have explicitly
configured one of these classes using <mergePolicy> and include an init arg
like this...
<bool name="useCompoundFile">true</bool>
...this will now be treated as if you specified...
<useCompoundFile>true</useCompoundFile>
...directly on the <indexConfig> (overriding any value already set using that
syntax) and a warning will be logged to updated your configuration. Users
with an explicitly declared <mergePolicy> are encouraged to review the
current javadocs for their MergePolicy subclass and review their configured
options carefully. See SOLR-4941, SOLR-4934 and LUCENE-5038 for more
information.
- SOLR-4778: The signature of LogWatcher.registerListener has changed, from
(ListenerConfig, CoreContainer) to (ListenerConfig). Users implementing their
own LogWatcher classes will need to change their code accordingly.
- LUCENE-5063: ByteField and ShortField have been deprecated and will be removed
in 5.0. If you are still using these field types, you should migrate your
fields to TrieIntField.
Detailed Change List
- New Features (22)
- SOLR-3251: Dynamically add fields to schema.
(Steve Rowe, Robert Muir, yonik)
- SOLR-4761, SOLR-4976: Add option to plugin a merged segment warmer into solrconfig.xml.
Info about segments warmed in the background is available via infostream.
(Mark Miller, Ryan Ernst, Mike McCandless, Robert Muir)
- SOLR-3240: Add "spellcheck.collateMaxCollectDocs" option so that when testing
potential Collations against the index, SpellCheckComponent will only collect
n documents, thereby estimating the hit-count. This is a performance optimization
in cases where exact hit-counts are unnecessary. Also, when "collateExtendedResults"
is false, this optimization is always made
(James Dyer).
- SOLR-4785: New MaxScoreQParserPlugin returning max() instead of sum() of terms
(janhoy)
- SOLR-4234: Add support for binary files in ZooKeeper.
(Eric Pugh via Mark Miller)
- SOLR-4048: Add findRecursive method to NamedList.
(Shawn Heisey)
- SOLR-4228: SolrJ's SolrPing object has new methods for ping, enable, and
disable.
(Shawn Heisey, hossman, Steve Rowe)
- SOLR-4893: Extend FieldMutatingUpdateProcessor.ConfigurableFieldNameSelector
to enable checking whether a field matches any schema field. To select field
names that don't match any fields or dynamic fields in the schema, add
<bool name="fieldNameMatchesSchemaField">false</bool> to an update
processor's configuration in solrconfig.xml.
(Steve Rowe, hossman)
- SOLR-4921: Admin UI now supports adding documents to Solr
(gsingers, steffkes)
- SOLR-4916: Add support to write and read Solr index files and transaction log
files to and from HDFS.
(phunt, Mark Miller, Gregory Chanan)
- SOLR-4892: Add FieldMutatingUpdateProcessorFactory subclasses
Parse{Date,Integer,Long,Float,Double,Boolean}UpdateProcessorFactory. These
factories have a default selector that matches all fields that either don’t
match any schema field, or are in the schema with the corresponding
typeClass. If they see a value that is not a CharSequence, or can't parse
the value, they leave it as is. For multi-valued fields, these processors
will not convert any values unless all are first successfully parsed, or
already are instances of the target class. Ordering the processors, e.g.
[Boolean, Long, Double, Date] will allow e.g. values ["2", "5", "8.6"] to
be left alone by the Boolean and Long processors, but then converted by the
Double processor.
(Steve Rowe, hossman)
- SOLR-4972: Add PUT command to ZkCli tool.
(Roman Shaposhnik via Mark Miller)
- SOLR-4973: Adding getter method for defaultCollection on CloudSolrServer.
(Furkan KAMACI via Mark Miller)
- SOLR-4897: Add solr/example/example-schemaless/, an example config set
for schemaless mode.
(Steve Rowe)
- SOLR-4655: Add option to have Overseer assign generic node names so that
new addresses can host shards without naming confusion.
(Mark Miller, Anshum Gupta)
- SOLR-4977: Add option to send IndexWriter's infostream to the logging system.
(Ryan Ernst via Robert Muir)
- SOLR-4693: A "deleteshard" collections API that unloads all replicas of a given
shard and then removes it from the cluster state. It will remove only those shards
which are INACTIVE or have no range (created for custom sharding).
(Anshum Gupta, shalin)
- SOLR-5003: CSV Update Handler supports optionally adding the line number/row id to
a document
(gsingers)
- SOLR-5010: Add support for creating copy fields to the Schema REST API
(gsingers)
- SOLR-4991: Register QParserPlugins as SolrInfoMBeans
(ehatcher)
- SOLR-4943: Add a new system wide info admin handler that exposes the system info
that could previously only be retrieved using a SolrCore.
(Mark Miller)
- SOLR-3076: Block joins. Documents and their sub-documents must be indexed
as a block.
{!parent which=<allParents>}<someChildren> takes in a query that matches child
documents and results in matches on their parents.
{!child of=<allParents>}<someParents> takes in a query that matches some parent
documents and results in matches on their children.
(Mikhail Khludnev, Vadim Kirilchuk, Alan Woodward, Tom Burton-West, Mike McCandless,
hossman, yonik)
- Bug Fixes (37)
- SOLR-4333: edismax parser to not double-escape colons if already escaped by
the client application
(James Dyer, Robert J. van der Boon)
- SOLR-4776: Solrj doesn't return "between" count in range facets
(Philip K. Warren via shalin)
- SOLR-4616: HitRatio on caches is now exposed over JMX MBeans as a float.
(Greg Bowyer)
- SOLR-4803: Fixed core discovery mode (ie: new style solr.xml) to treat
'collection1' as the default core name.
(hossman)
- SOLR-4790: Throw an error if a core has the same name as another core, both old and
new style solr.xml
- SOLR-4842: Fix facet.field local params from affecting other facet.field's.
(ehatcher, hossman)
- SOLR-4814: If a SolrCore cannot be created it should remove any information it
published about itself from ZooKeeper.
(Mark Miller)
- SOLR-4863: Removed non-existent attribute sourceId from dynamic JMX stats
to fix AttributeNotFoundException
(suganuma, hossman via shalin)
- SOLR-4891: JsonLoader should preserve field value types from the JSON content stream.
(Steve Rowe)
- SOLR-4805: SolrCore#reload should not call preRegister and publish a DOWN state to
ZooKeeper.
(Mark Miller, Jared Rodriguez)
- SOLR-4899: When reconnecting after ZooKeeper expiration, we need to be willing to wait
forever, not just for 30 seconds.
(Mark Miller)
- SOLR-4920: JdbcDataSource incorrectly suppresses exceptions when retrieving a connection from
a JNDI context and falls back to trying to use DriverManager to obtain a connection. Additionally,
if a SQLException is thrown while initializing a connection, such as in setAutoCommit(), the
connection will not be closed.
(Chris Eldredge via shalin)
- SOLR-4915: The root cause should be returned to the user when a SolrCore create call fails.
(Mark Miller)
- SOLR-4925 : Collection create throws NPE when 'numShards' param is missing
(Noble Paul)
- SOLR-4910: persisting solr.xml is broken. More stringent testing of persistence fixed
up a number of issues and several bugs with persistence. Among them are
-
don't persisting implicit properties
-
should persist zkHost in the <solr> tag (user's list)
-
reloading a core that has transient="true" returned an error. reload should load
a transient core if it's not yet loaded.
-
No longer persisting loadOnStartup or transient core properties if they were not
specified in the original solr.xml
-
Testing flushed out the fact that you couldn't swap a core marked transient=true
loadOnStartup=false because it hadn't been loaded yet.
-
SOLR-4862, CREATE fails to persist schema, config, and dataDir
-
SOLR-4363, not persisting coreLoadThreads in <solr> tag
-
SOLR-3900, logWatcher properties not persisted
-
SOLR-4850, cores defined as loadOnStartup=true, transient=false can't be searched
(Erick Erickson)
- SOLR-4923: Commits to non leaders as part of a request that also contain updates
can execute out of order.
(hossman, Ricardo Merizalde, Mark Miller)
- SOLR-4932: persisting solr.xml saves some parameters it shouldn't when they weren't
defined in the original. Benign since the default values are saved, but still incorrect.
(Erick Erickson, thanks Shawn Heisey for helping test!)
- SOLR-4934, SOLR-4941: Fix handling of <mergePolicy> init arg
"useCompoundFile" needed after changes in LUCENE-5038
(hossman)
- SOLR-4456: Admin UI: Displays dashboard even if Solr is down
(steffkes)
- SOLR-4949: UI Analysis page dropping characters from input box
(steffkes)
- SOLR-4960: Fix race conditions in shutdown of CoreContainer
and getCore that could cause a request to attempt to use a core that
has shut down.
(yonik)
- SOLR-4926: Fixed rare replication bug that normally only manifested when
using compound file format.
(yonik, Mark Miller)
- SOLR-4974: Outgrowth of SOLR-4960 that includes transient cores and pending cores
(Erick Erickson)
- SOLR-3369: shards.tolerant=true is broken for group queries
(Russell Black, Martijn van Groningen, Jabouille jean Charles, Ryan McKinley via shalin)
- SOLR-4452: Hunspell stemmer should not merge duplicate dictionary entries
(janhoy)
- SOLR-5000: ManagedIndexSchema doesn't persist uniqueKey tag after calling addFields
method.
(Jun Ohtani, Steve Rowe)
- SOLR-4982: Creating a core while referencing system properties looks like it loses files
Actually, instanceDir, config, dataDir and schema are not dereferenced properly
when creating cores that reference sys vars (e.g. &dataDir=${dir}). In the dataDir
case in particular this leads to the index being put in a directory literally named
${dir} but on restart the sysvar will be properly dereferenced.
- SOLR-4788: Multiple Entities DIH delta import: dataimporter.[entityName].last_index_time
is empty.
(chakming wong, James Dyer via shalin)
- SOLR-4978: Time is stripped from datetime column when imported into Solr date field
if convertType=true.
(Bill Au, shalin)
- SOLR-5019: spurious ConcurrentModificationException when spell check component
was in use with filters.
(yonik)
- SOLR-5018: The Overseer should avoid publishing the state for collections that do not
exist under the /collections zk node.
(Mark Miller)
- SOLR-5028,SOLR-5029: ShardHandlerFactory was not being created properly when
using new-style solr.xml, and was not being persisted properly when using
old-style.
(Tomás Fernández Löbbe, Ryan Ernst, Alan Woodward)
- SOLR-4997: The splitshard api doesn't call commit on new sub shards before
switching shard states. Multiple bugs related to sub shard recovery and
replication are also fixed.
(shalin)
- SOLR-5034: A facet.query that parses or analyzes down to a null Query would
throw a NPE. Fixed.
(David Smiley)
- SOLR-5039: Admin/Schema Browser displays -1 for term counts for multiValued fields.
- SOLR-5037: The CSV loader now accepts field names that are not in the schema.
(gsingers, ehatcher, Steve Rowe)
- SOLR-4791: solr.xml sharedLib does not work in 4.3.0
(Ryan Ernst, Jan Høydahl via
Erick Erickson)
- Optimizations (9)
- SOLR-4923: Commit to all nodes in a collection in parallel rather than locally and
then to all other nodes.
(hossman, Ricardo Merizalde, Mark Miller)
- SOLR-3838: Admin UI - Multiple filter queries are not supported in Query UI
(steffkes)
- SOLR-4719 : Admin UI - Default to wt=json on Query-Screen
(steffkes)
- SOLR-4611: Admin UI - Analysis-Urls with empty parameters create empty result table
(steffkes)
- SOLR-4955: Admin UI - Show address bar on top for Schema + Config
(steffkes)
- SOLR-4412: New parameter langid.lcmap to map detected language code to be placed
in "language" field
(janhoy)
- SOLR-4815: Admin-UI - DIH: Let "commit" be checked by default
(steffkes)
- SOLR-5002: optimize numDocs(Query,DocSet) when filterCache is null
(Robert Muir)
- SOLR-5012: optimize search with filter when filterCache is null
(Robert Muir)
- Other Changes (17)
- SOLR-4737: Update Guava to 14.0.1
(Mark Miller)
- SOLR-2079: Add option to pass HttpServletRequest in the SolrQueryRequest context map.
(Tomás Fernández Löbbe via Robert Muir)
- SOLR-4738: Update Jetty to 8.1.10.v20130312
(Mark Miller, Robert Muir)
- SOLR-4749: Clean up and refactor CoreContainer code around solr.xml and SolrCore
management.
(Mark Miller)
- SOLR-4547: Move logging of filenames on commit from INFO to DEBUG.
(Shawn Heisey, hossman)
- SOLR-4757: Change the example to use the new solr.xml format and core
discovery by directory structure.
(Mark Miller)
- SOLR-4759: Velocity (/browse) template cosmetic cleanup.
(Mark Bennett, ehatcher)
- SOLR-4778: LogWatcher init code moved out of CoreContainer
(Alan Woodward)
- SOLR-4784: Make class LuceneQParser public
(janhoy)
- SOLR-4448: Allow the solr internal load balancer to be more easily pluggable.
(Philip Hoy via Robert Muir)
- SOLR-4224: Refactor JavaBinCodec input stream definition to enhance reuse.
(phunt via Mark Miller)
- SOLR-4931: SolrDeletionPolicy onInit and onCommit methods changed to override
exact signatures (with generics) from IndexDeletionPolicy
(shalin)
- SOLR-4942: test improvements to randomize use of compound files
(hossman)
- SOLR-4966: CSS, JS and other files in webapp without license
(uschindler,
steffkes)
- SOLR-4986: Upgrade to Tika 1.4
(Markus Jelsma via janhoy)
- SOLR-4948, SOLR-5009: Tidied up CoreContainer construction logic.
(Alan Woodward, Uwe Schindler, Steve Rowe)
- LUCENE-5107: Properties files by Solr are now written in UTF-8 encoding,
Unicode is no longer escaped. Reading of legacy properties files with
\u escapes is still possible.
(Uwe Schindler, Robert Muir)
- Versions of Major Components (5)
- Apache Tika 1.3
- Carrot2 3.6.2
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.5
Detailed Change List
- Bug Fixes (22)
- SOLR-4795: Sub shard leader should not accept any updates from parent after
it goes active
(shalin)
- SOLR-4798: shard splitting does not respect the router for the collection
when executing the index split. One effect of this is that documents
may be placed in the wrong shard when the default compositeId router
is used in conjunction with IDs containing "!".
(yonik)
- SOLR-4797: Shard splitting creates sub shards which have the wrong hash
range in cluster state. This happens when numShards is not a power of two
and router is compositeId.
(shalin)
- SOLR-4806: Shard splitting does not abort if WaitForState times out
(shalin)
- SOLR-4807: The zkcli script now works with log4j. The zkcli.bat script
was broken on Windows in 4.3.0, now it works.
(Shawn Heisey)
- SOLR-4813: Fix SynonymFilterFactory to allow init parameters for
tokenizer factory used when parsing synonyms file.
(Shingo Sasaki, hossman)
- SOLR-4829: Fix transaction log leaks (a failure to clean up some old logs)
on a shard leader, or when unexpected exceptions are thrown during log
recovery.
(Steven Bower, Mark Miller, yonik)
- SOLR-4751: Fix replication problem of files in sub directory of conf directory.
(Minoru Osuka via Koji)
- SOLR-4741: Deleting a collection should set DELETE_DATA_DIR to true.
(Mark Miller)
- SOLR-4752: There are some minor bugs in the Collections API parameter
validation.
(Mark Miller)
- SOLR-4563: RSS DIH-example not working
(janhoy)
- SOLR-4796: zkcli.sh should honor JAVA_HOME
(Roman Shaposhnik via Mark Miller)
- SOLR-4734: Leader election fails with an NPE if there is no UpdateLog.
(Mark Miller, Alexander Eibner)
- SOLR-4868: Setting the log level for the log4j root category results in
adding a new category, the empty string.
(Shawn Heisey)
- SOLR-4855: DistributedUpdateProcessor doesn't check for peer sync requests
(shalin)
- SOLR-4867: Admin UI - setting loglevel on root throws RangeError
(steffkes)
- SOLR-4870: RecentUpdates.update() does not increment numUpdates loop counter
(Alexey Kudinov via shalin)
- SOLR-4877, LUCENE-5023: Removed SolrIndexSearcher#getDocSetNC()'s special
case for handling TermQuery to prevent NullPointerException if reader does
not have fields.
(Bao Yang Yang, Uwe Schindler)
- SOLR-4881: Fix DocumentAnalysisRequestHandler to correctly use
EmptyEntityResolver to prevent loading of external entities like
UpdateRequestHandler does.
(Hossman, Uwe Schindler)
- SOLR-4858: SolrCore reloading was broken when the UpdateLog
was enabled.
(Hossman, Anshum Gupta, Alexey Serba, Mark Miller, yonik)
- SOLR-4853: Fixed SolrJettyTestBase so it may be reused by end users
(hossman)
- SOLR-4744: Update failure on sub shard is not propagated to clients by parent
shard
(Anshum Gupta, yonik, shalin)
- Other Changes (1)
- SOLR-4760: Include core name in logs when loading schema.
(Shawn Heisey)
- Versions of Major Components (5)
- Apache Tika 1.3
- Carrot2 3.6.2
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.5
- Upgrading from Solr 4.2.0 (3)
- In the schema REST API, the output path for copyFields and dynamicFields
has been changed from all lowercase "copyfields" and "dynamicfields" to
camelCase "copyFields" and "dynamicFields", respectively, to align with all
other schema REST API outputs, which use camelCase. The URL format remains
the same: all resource names are lowercase. See SOLR-4623 for details.
- Slf4j/logging jars are no longer included in the Solr webapp. All logging
jars are now in example/lib/ext. Changing logging impls is now as easy as
updating the jars in this folder with those necessary for the logging impl
you would like. If you are using another webapp container, these jars will
need to go in the corresponding location for that container.
In conjunction, the dist-excl-slf4j and dist-war-excl-slf4 build targets
have been removed since they are redundant. See the Slf4j documentation,
SOLR-3706, and SOLR-4651 for more details.
- The hardcoded SolrCloud defaults for 'hostContext="solr"' and
'hostPort="8983"' have been deprecated and will be removed in Solr 5.0.
Existing solr.xml files that do not have these options explicitly specified
should be updated accordingly. See SOLR-4622 for more details.
Detailed Change List
- New Features (11)
- SOLR-4648 PreAnalyzedUpdateProcessorFactory allows using the functionality
of PreAnalyzedField with other field types. See javadoc for details and
examples.
(Andrzej Bialecki)
- SOLR-4623: Provide REST API read access to all elements of the live schema.
Add a REST API request to return the entire live schema, in JSON, XML, and
schema.xml formats. Move REST API methods from package org.apache.solr.rest
to org.apache.solr.rest.schema, and rename base functionality REST API
classes to remove the current schema focus, to prepare for other non-schema
REST APIs. Change output path for copyFields and dynamicFields from
"copyfields" and "dynamicfields" (all lowercase) to "copyFields" and
"dynamicFields", respectively, to align with all other REST API outputs, which
use camelCase.
(Steve Rowe)
- SOLR-4658: In preparation for REST API requests that can modify the schema,
a "managed schema" is introduced.
Add '<schemaFactory class="ManagedSchemaFactory" mutable="true"/>' to solrconfig.xml
in order to use it, and to enable schema modifications via REST API requests.
(Steve Rowe, Robert Muir)
- SOLR-4656: Added two new highlight parameters, hl.maxMultiValuedToMatch and
hl.maxMultiValuedToExamine. maxMultiValuedToMatch stops looking for snippets after
finding the specified number of matches, no matter how far into the multivalued field
you've gone. maxMultiValuedToExamine stops looking for matches after the specified
number of multiValued entries have been examined. If both are specified, the limit
hit first stops the loop. Also this patch cuts down on the copying of the document
entries during highlighting. These optimizations are probably unnoticeable unless
there are a large number of entries in the multiValued field. Conspicuously, this will
prevent the "best" match from being found if it appears later in the MV list than the
cutoff specified by either of these params.
(Erick Erickson)
- SOLR-4675: Improve PostingsSolrHighlighter to support per-field/query-time overrides
and add additional configuration parameters. See the javadocs for more details and
examples.
(Robert Muir)
- SOLR-3755: A new collections api to add additional shards dynamically by splitting
existing shards.
(yonik, Anshum Gupta, shalin)
- SOLR-4530: DIH: Provide configuration to use Tika's IdentityHtmlMapper
(Alexandre Rafalovitch via shalin)
- SOLR-4662: Discover SolrCores by directory structure rather than defining them
in solr.xml. Also, change the format of solr.xml to be closer to that of solrconfig.xml.
This version of Solr will ship the example in the old style, but you can manually
try the new style. Solr 4.4 will ship with the new style, and Solr 5.0 will remove
support for the old style.
(Erick Erickson, Mark Miller)
Additional Work:
-
SOLR-4347: Ensure that newly-created cores via Admin handler are persisted in solr.xml
(Erick Erickson)
-
SOLR-1905: Cores created by the admin request handler should be persisted to solr.xml.
Also fixed a problem whereby properties like solr.solr.datadir would be persisted
to solr.xml. Also, cores that didn't happen to be loaded were not persisted.
(Erick Erickson)
- SOLR-4717/SOLR-1351: SimpleFacets now work with localParams allowing faceting on the
same field multiple ways
(ryan, Uri Boness)
- SOLR-4671: CSVResponseWriter now supports pseudo fields.
(ryan, nihed mbarek)
- SOLR-4358: HttpSolrServer sends the stream name and exposes 'useMultiPartPost'
(Karl Wright via ryan)
- Bug Fixes (23)
- SOLR-4543: setting shardHandlerFactory in solr.xml/solr.properties does not work.
(Ryan Ernst, Robert Muir via Erick Erickson)
- SOLR-4634: Fix scripting engine tests to work with Java 8's "Nashorn" Javascript
implementation.
(Uwe Schindler)
- SOLR-4636: If opening a reader fails for some reason when opening a SolrIndexSearcher,
a Directory can be left unreleased.
(Mark Miller)
- SOLR-4405: Admin UI - admin-extra files are not rendered into the core-menu
(steffkes)
- SOLR-3956: Fixed group.facet=true to work with negative facet.limit
(Chris van der Merwe, hossman)
- SOLR-4650: copyField doesn't work with source globs that don't match any
explicit or dynamic fields. This regression was introduced in Solr 4.2.
(Daniel Collins, Steve Rowe)
- SOLR-4641: Schema now throws exception on illegal field parameters.
(Robert Muir)
- SOLR-3758: Fixed SpellCheckComponent to work consistently with distributed grouping
(James Dyer)
- SOLR-4652: Fix broken behavior with shared libraries in resource loader for
solr.xml plugins.
(Ryan Ernst, Robert Muir, Uwe Schindler)
- SOLR-4664: ZkStateReader should update aliases on construction.
(Mark Miller, Elodie Sannier)
- SOLR-4682: CoreAdminRequest.mergeIndexes can not merge multiple cores or indexDirs.
(Jason.D.Cao via shalin)
- SOLR-4581: When faceting on numeric fields in Solr 4.2, negative values (constraints)
were sorted incorrectly.
(Alexander Buhr, shalin, yonik)
- SOLR-4699: The System admin handler should not assume a file system based data directory
location.
(Mark Miller)
- SOLR-4695: Fix core admin SPLIT action to be useful with non-cloud setups
(shalin)
- SOLR-4680: Correct example spellcheck configuration's queryAnalyzerFieldType and
use "text" field instead of narrower "name" field
(ehatcher, Mark Bennett)
- SOLR-4702: Fix example /browse "Did you mean?" suggestion feature.
(ehatcher, Mark Bennett)
- SOLR-4710: You cannot delete a collection fully from ZooKeeper unless all nodes are up and
functioning correctly.
(Mark Miller)
- SOLR-4487: SolrExceptions thrown by HttpSolrServer will now contain the
proper HTTP status code returned by the remote server, even if that status
code is not something Solr itself returned -- eg: from the Servlet Container,
or an intermediate HTTP Proxy
(hossman)
- SOLR-4661: Admin UI Replication details now correctly displays the current
replicable generation/version of the master.
(hossman)
- SOLR-4716,SOLR-4584: SolrCloud request proxying does not work on Tomcat and
perhaps other non Jetty containers.
(Po Rui, Yago Riveiro via Mark Miller)
- SOLR-4746: Distributed grouping used a NamedList instead of a SimpleOrderedMap
for the top level group commands, causing output formatting differences
compared to non-distributed grouping.
(yonik)
- SOLR-4705: Fixed bug causing NPE when querying a single replica in SolrCloud
using the shards param
(Raintung Li, hossman)
- SOLR-4729: LukeRequestHandler: Using a dynamic copyField source that is
not also a dynamic field triggers error message 'undefined field: "(glob)"'.
(Adam Hahn, hossman, Steve Rowe)
- Optimizations (none)
- Other Changes (11)
- SOLR-4653: Solr configuration should log inaccessible/ non-existent relative paths in lib
dir=...
(Dawid Weiss)
- SOLR-4317: SolrTestCaseJ4: Can't avoid "collection1" convention
(Tricia Jenkins, via Erick Erickson)
- SOLR-4571: SolrZkClient#setData should return Stat object.
(Mark Miller)
- SOLR-4603: CachingDirectoryFactory should use an IdentityHashMap for
byDirectoryCache.
(Mark Miller)
- SOLR-4544: Refactor HttpShardHandlerFactory so load-balancing logic can be customized.
(Ryan Ernst via Robert Muir)
- SOLR-4607: Use noggit 0.5 release jar rather than a forked copy.
(Yonik Seeley, Robert Muir)
- SOLR-3706: Ship setup to log with log4j.
(ryan, Mark Miller)
- SOLR-4651: Remove dist-excl-slf4j build target.
(Shawn Heisey)
- SOLR-4622: The hardcoded SolrCloud defaults for 'hostContext="solr"' and
'hostPort="8983"' have been deprecated and will be removed in Solr 5.0.
Existing solr.xml files that do not have these options explicitly specified
should be updated accordingly.
(hossman)
- SOLR-4672: Requests attempting to use SolrCores which had init failures
(that would be reported by CoreAdmin STATUS requests) now result in 500
error responses with the details about the init failure, instead of 404
error responses.
(hossman)
- SOLR-4730: Make the wiki link more prominent in the release documentation.
(Uri Laserson via Robert Muir)
- Versions of Major Components (5)
- Apache Tika 1.3
- Carrot2 3.6.2
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.5
Detailed Change List
- Bug Fixes (38)
- SOLR-4567: copyField source glob matching explicit field(s) stopped working
in Solr 4.2.
(Alexandre Rafalovitch, Steve Rowe)
- SOLR-4475: Fix various places that still assume File based paths even when
not using a file based DirectoryFactory.
(Mark Miller)
- SOLR-4551: CachingDirectoryFactory needs to create CacheEntry's with the
fullpath not path.
(Mark Miller)
- SOLR-4555: When forceNew is used with CachingDirectoryFactory#get, the old
CacheValue should give up its path as it will be used by a new Directory
instance.
(Mark Miller)
- SOLR-4578: CoreAdminHandler#handleCreateAction gets a SolrCore and does not
close it in SolrCloud mode when a core with the same name already exists.
(Mark Miller)
- SOLR-4574: The Collections API will silently return success on an unknown
ACTION parameter.
(Mark Miller)
- SOLR-4576: Collections API validation errors should cause an exception on
clients and otherwise act as validation errors with the Core Admin API.
(Mark Miller)
- SOLR-4577: The collections API should return responses (success or failure)
for each node it attempts to work with.
(Mark Miller)
- SOLR-4568: The lastPublished state check before becoming a leader is not
working correctly.
(Mark Miller)
- SOLR-4570: Even if an explicit shard id is used, ZkController#preRegister
should still wait to see the shard id in its current ClusterState.
(Mark Miller)
- SOLR-4585: The Collections API validates numShards with < 0 but should use
<= 0.
(Mark Miller)
- SOLR-4592: DefaultSolrCoreState#doRecovery needs to check the CoreContainer
shutdown flag inside the recoveryLock sync block.
(Mark Miller)
- SOLR-4595: CachingDirectoryFactory#close can throw a concurrent
modification exception.
(Mark Miller)
- SOLR-4573: Accessing Admin UI files in SolrCloud mode logs warnings.
(Mark Miller, Phil John)
- SOLR-4594: StandardDirectoryFactory#remove accesses byDirectoryCache
without a lock.
(Mark Miller)
- SOLR-4597: CachingDirectoryFactory#remove should not attempt to empty/remove
the index right away but flag for removal after close.
(Mark Miller)
- SOLR-4598: The Core Admin unload command's option 'deleteDataDir', should use
the DirectoryFactory API to remove the data dir.
(Mark Miller)
- SOLR-4599: CachingDirectoryFactory calls close(Directory) on forceNew if the
Directory has a refCnt of 0, but it should call closeDirectory(CacheValue).
(Mark Miller)
- SOLR-4602: ZkController#unregister should cancel its election participation
before asking the Overseer to delete the SolrCore information.
(Mark Miller)
- SOLR-4601: A Collection that is only partially created and then deleted will
leave pre allocated shard information in ZooKeeper.
(Mark Miller)
- SOLR-4604: UpdateLog#init is over called on SolrCore#reload.
(Mark Miller)
- SOLR-4605: Rollback does not work correctly.
(Mark S, Mark Miller)
- SOLR-4609: The Collections API should only send the reload command to ACTIVE
cores.
(Mark Miller)
- SOLR-4297: Atomic update request containing null=true sets all subsequent
fields to null
(Ben Pennell, Rob, shalin)
- SOLR-4371: Admin UI - Analysis Screen shows empty result
(steffkes)
- SOLR-4318: NPE encountered with querying with wildcards on a field that uses
the DefaultAnalyzer (i.e. no analysis chain defined).
(Erick Erickson)
- SOLR-4361: DataImportHandler would throw UnsupportedOperationException if
handler-level parameters were specified containing periods in the name
(James Dyer)
- SOLR-4538: Date Math expressions were being truncated to 32 characters
when used in field:value queries in the lucene QParser.
(hossman, yonik)
- SOLR-4617: SolrCore#reload needs to pass the deletion policy to the next
SolrCore through its constructor rather than setting a field after.
(Mark Miller)
- SOLR-4589: Fixed CPU spikes and poor performance in lazy field loading
of multivalued fields.
(hossman)
- SOLR-4608: Update Log replay and PeerSync replay should use the default
processor chain to update the index.
(Ludovic Boutros, yonik)
- SOLR-4625: The solr (lucene syntax) query parser lost top-level boost
values and top-level phrase slops on queries produced by nested
sub-parsers.
(yonik)
- SOLR-4624: CachingDirectoryFactory does not need to support forceNew any
longer and it appears to be causing a missing close directory bug. forceNew
is no longer respected and will be removed in 4.3.
(Mark Miller)
- SOLR-3819: Grouped faceting (group.facet=true) did not respect filter
exclusions.
(Petter Remen, yonik)
- SOLR-4637: Replication can sometimes wait until shutdown or core unload until
removing some tmp directories.
(Mark Miller)
- SOLR-4638: DefaultSolrCoreState#getIndexWriter(null) is a way to avoid
creating the IndexWriter earlier than necessary, but it's not
implemented quite right.
(Mark Miller)
- SOLR-4640: CachingDirectoryFactory can fail to close directories in some race
conditions.
(Mark Miller)
- SOLR-4642: QueryResultKey is not calculating the correct hashCode for filters.
(Joel Bernstein via Mark Miller)
- Optimizations (2)
- SOLR-4569: waitForReplicasToComeUp should bail right away if it doesn't see the
expected slice in the clusterstate rather than waiting.
(Mark Miller)
- SOLR-4311: Admin UI - Optimize Caching Behaviour
(steffkes)
- Other Changes (2)
- SOLR-4537: Clean up schema information REST API.
(Steve Rowe)
- SOLR-4596: DistributedQueue should ensure its full path exists in the constructor.
(Mark Miller)
- Versions of Major Components (5)
- Apache Tika 1.3
- Carrot2 3.6.2
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.5
- Upgrading from Solr 4.1.0 (1)
- (No upgrade instructions yet)
Detailed Change List
- New Features (15)
- SOLR-4043: Add ability to get success/failure responses from Collections API.
(Raintung Li, Mark Miller)
- SOLR-2827: RegexpBoost Update Processor
(janhoy)
- SOLR-4370: Allow configuring commitWithin to do hard commits.
(Mark Miller, Senthuran Sivananthan)
- SOLR-4451: SolrJ, and SolrCloud internals, now use SystemDefaultHttpClient
under the covers -- allowing many HTTP connection related properties to be
controlled via 'standard' java system properties.
(hossman)
- SOLR-3855, SOLR-4490: Doc values support.
(Adrien Grand, Robert Muir)
- SOLR-4417: Reopen the IndexWriter on SolrCore reload.
(Mark Miller)
- SOLR-4477: Add support for queries (match-only) against docvalues fields.
(Robert Muir)
- SOLR-4488: Return slave replication details for a master if the master has
also acted like a slave.
(Mark Miller)
- SOLR-4498: Add list command to ZkCLI that prints out the contents of
ZooKeeper.
(Roman Shaposhnik via Mark Miller)
- SOLR-4481: SwitchQParserPlugin registered by default as 'switch' using
syntax: {!switch case=XXX case.foo=YYY case.bar=ZZZ default=QQQ}foo
(hossman)
- SOLR-4078: Allow custom naming of SolrCloud nodes so that a new host:port
combination can take over for a previous shard.
(Mark Miller)
- SOLR-4210: Requests to a Collection that does not exist on the receiving node
should be proxied to a suitable node.
(Mark Miller, Po Rui, yonik)
- SOLR-1365: New SweetSpotSimilarityFactory allows customizable TF/IDF based
Similarity when you know the optimal "Sweet Spot" of values for the field
length and TF scoring factors.
(hossman)
- SOLR-4138: CurrencyField fields can now be used in a ValueSources to
get the "raw" value (using the default number of fractional digits) in
the default currency of the field type. There is also a new
currency(field,[CODE]) function for generating a ValueSource of the
"natural" value, converted to an optionally specified currency to
override the default for the field type.
(hossman)
- SOLR-4503: Add REST API methods, via Restlet integration, for reading schema
elements, at /schema/fields/, /schema/dynamicfields/, /schema/fieldtypes/,
and /schema/copyfields/.
(Steve Rowe)
- Bug Fixes (34)
- SOLR-2850: Do not refine facets when minCount == 1
(Matt Smith, lundgren via Adrien Grand)
- SOLR-4309: /browse: Improve JQuery autosuggest behavior
(janhoy)
- SOLR-4330: group.sort is ignored when using group.truncate and ex/tag
local params together
(koji)
- SOLR-4321: Collections API will sometimes use a node more than once, even
when more unused nodes are available.
(Eric Falcao, Brett Hoerner, Mark Miller)
- SOLR-4345 : Solr Admin UI doesn't work in IE 10
(steffkes)
- SOLR-4349 : Admin UI - Query Interface does not work in IE
(steffkes)
- SOLR-4359: The RecentUpdates#update method should treat a problem reading the
next record the same as a problem parsing the record - log the exception and
break.
(Mark Miller)
- SOLR-4225: Term info page under schema browser shows incorrect count of terms
(steffkes)
- SOLR-3926: Solr should support better way of finding active sorts
(Eirik Lygre via
Erick Erickson)
- SOLR-4342: Fix DataImportHandler stats to be a proper Map
(hossman)
- SOLR-3967: langid.enforceSchema option checks source field instead of target field
(janhoy)
- SOLR-4380: Replicate after startup option would not replicate until the
IndexWriter was lazily opened.
(Mark Miller, Gregg Donovan)
- SOLR-4400: Deadlock can occur in a rare race between committing and
closing a SolrIndexWriter.
(Erick Erickson, Mark Miller)
- SOLR-3655: A restarted node can briefly appear live and active before it really
is in some cases.
(Mark Miller)
- SOLR-4426: NRTCachingDirectoryFactory does not initialize maxCachedMB and maxMergeSizeMB
if <directoryFactory> is not present in solrconfig.xml
(Jack Krupansky via shalin)
- SOLR-4463: Fix SolrCoreState reference counting.
(Mark Miller)
- SOLR-4459: The Replication 'index move' rather than copy optimization doesn't
kick in when using NRTCachingDirectory or the rate limiting feature.
(Mark Miller)
- SOLR-4421,SOLR-4165: On CoreContainer shutdown, all SolrCores should publish their
state as DOWN.
(Mark Miller, Markus Jelsma)
- SOLR-4467: Ephemeral directory implementations may not recover correctly
because the code to clear the tlog files on startup is off.
(Mark Miller)
- SOLR-4413: Fix SolrCore#getIndexDir() to return the current index directory.
(Gregg Donovan, Mark Miller)
- SOLR-4469: A new IndexWriter must be opened on SolrCore reload when the index
directory has changed and the previous SolrCore's state should not be
propagated.
(Mark Miller, Gregg Donovan)
- SOLR-4471: Replication occurs even when a slave is already up to date.
(Mark Miller, Andre Charton)
- SOLR-4484: ReplicationHandler#loadReplicationProperties still uses Files
rather than the Directory to try and read the replication properties files.
(Mark Miller)
- SOLR-4352: /browse pagination now supports and preserves sort context
(Eric Spiegelberg, Erik Hatcher)
- LUCENE-4796, SOLR-4373: Fix concurrency issue in NamedSPILoader and
AnalysisSPILoader when doing concurrent core loads in multicore
Solr configs.
(Uwe Schindler, Hossman)
- SOLR-4504: Fixed CurrencyField range queries to correctly exclude
documents w/o values
(hossman)
- SOLR-4480: A trailing + or - caused the edismax parser to throw
an exception.
(Fiona Tay, Jan Høydahl, yonik)
- SOLR-4507: The Cloud tab does not show up in the Admin UI if you
set zkHost in solr.xml.
(Alfonso Presa, Mark Miller)
- SOLR-4505: Possible deadlock around SolrCoreState update lock.
(Erick Erickson, Mark Miller)
- SOLR-4511: When a new index is replicated into place, we need
to update the most recent replicatable index point without
doing a commit. This is important for repeater use cases, as
well as when nodes may switch master/slave roles.
(Mark Miller, Raúl Grande)
- SOLR-4515: CurrencyField's OpenExchangeRatesOrgProvider now requires
a ratesFileLocation init param, since the previous global default
no longer works
(hossman)
- SOLR-4518: Improved CurrencyField error messages when attempting to
use a Currency that is not supported by the current JVM.
(hossman)
- SOLR-3798: Fix copyField implementation in IndexSchema to handle
dynamic field references that aren't string-equal to the name of
the referenced dynamic field.
(Steve Rowe)
- SOLR-4497: Collection Aliasing.
(Mark Miller)
- Optimizations (10)
- SOLR-4339: Admin UI - Display Field-Flags on Schema-Browser
(steffkes)
- SOLR-4340: Admin UI - Analysis's Button Spinner goes wild
(steffkes)
- SOLR-4341: Admin UI - Plugins/Stats Page contains loooong
Values which result in horizontal Scrollbar
(steffkes)
- SOLR-3915: Color Legend for Cloud UI
(steffkes)
- SOLR-4306: Utilize indexInfo=false when gathering core names in UI
(steffkes)
- SOLR-4284: Admin UI - make core list scrollable separate from the rest of
the UI
(steffkes)
- SOLR-4364: Admin UI - Locale based number formatting
(steffkes)
- SOLR-4521: Stop using the 'force' option for recovery replication. This
will keep some less common unnecessary replications from happening.
(Mark Miller, Simon Scofield)
- SOLR-4529: Improve Admin UI Dashboard legibility
(Felix Buenemann via
steffkes)
- SOLR-4526: Admin UI depends on optional system info
(Felix Buenemann via
steffkes)
- Other Changes (15)
- SOLR-4259: Carrot2 dependency should be declared on the mini version, not the core.
(Dawid Weiss).
- SOLR-4348: Make the lock type configurable by system property by default.
(Mark Miller)
- SOLR-4353: Renamed example jetty context file to reduce confusion
(hossman)
- SOLR-4384: Make post.jar report timing information
(Upayavira via janhoy)
- SOLR-4415: Add 'state' to shards (default to 'active') and read/write them to
ZooKeeper
(Anshum Gupta via shalin)
- SOLR-4394: Tests and example configs demonstrating SSL with both server
and client certs
(hossman)
- SOLR-3060: SurroundQParserPlugin highlighting tests
(Ahmet Arslan via hossman)
- SOLR-2470: Added more tests for VelocityResponseWriter
- SOLR-4471: Improve and clean up TestReplicationHandler.
(Amit Nithian via Mark Miller)
- SOLR-3843: Include lucene codecs jar and enable per-field postings and docvalues
support in the schema.xml
(Robert Muir, Steve Rowe)
- SOLR-4511: Add new test for 'repeater' replication node.
(Mark Miller)
- SOLR-4458: Sort directions (asc, desc) are now case insensitive
(Shawn Heisey via hossman)
- SOLR-2996: A bare * without a field specification is treated as *:*
by the lucene and edismax query parsers.
(hossman, Jan Høydahl, Alan Woodward, yonik)
- SOLR-4416: Upgrade to Tika 1.3.
(Markus Jelsma via Mark Miller)
- SOLR-4200: Reduce INFO level logging from CachingDirectoryFactory
(Shawn Heisey via hossman)
- Versions of Major Components (5)
- Apache Tika 1.2
- Carrot2 3.6.2
- Velocity 1.7 and Velocity Tools 2.0
- Apache UIMA 2.3.1
- Apache ZooKeeper 3.4.5
- Upgrading from Solr 4.0.0 (2)
- Custom java parsing plugins need to migrate from throwing the internal
ParseException to throwing SyntaxError.
- BaseDistributedSearchTestCase now randomizes the servlet context it uses when
creating Jetty instances. Subclasses that assume a hard coded context of
"/solr" should either be fixed to use the "String context" variable, or should
take advantage of the new BaseDistributedSearchTestCase(String) constructor
to explicitly specify a fixed servlet context path. See SOLR-4136 for details.
Detailed Change List
- New Features (35)
- SOLR-2255: Enhanced pivot faceting to use local-params in the same way that
regular field value faceting can. This means support for excluding a filter
query, using a different output key, and specifying 'threads' to do
facet.method=fcs concurrently. PivotFacetHelper now extends SimpleFacet and
the getFacetImplementation() extension hook was removed.
(dsmiley)
- SOLR-3897: A highlighter parameter "hl.preserveMulti" to return all of the
values of a multiValued field in their original order when highlighting.
(Joel Bernstein via yonik)
- SOLR-3929: Support configuring IndexWriter max thread count in solrconfig.
(phunt via Mark Miller)
- SOLR-3906: Add support for AnalyzingSuggester (LUCENE-3842), where the
underlying analyzed form used for suggestions is separate from the returned
text.
(Robert Muir)
- SOLR-3985: ExternalFileField caches can be reloaded on firstSearcher/
newSearcher events using the ExternalFileFieldReloader
(Alan Woodward)
- SOLR-3911: Make Directory and DirectoryFactory first class so that the majority
of Solr's features work with any custom implementations.
(Mark Miller)
Additional Work:
-
SOLR-4032: Files larger than an internal buffer size fail to replicate.
(Mark Miller, Markus Jelsma)
-
SOLR-4033: Consistently use the solrconfig.xml lockType everywhere.
(Mark Miller, Markus Jelsma)
-
SOLR-4144: Replication using too much RAM.
(yonik, Markus Jelsma)
-
SOLR-4187: NPE on Directory release
(Mark Miller, Markus Jelsma)
- SOLR-4051: Add <propertyWriter /> element to DIH's data-config.xml file,
allowing the user to specify the location, filename and Locale for
the "data-config.properties" file. Alternatively, users can specify their
own property writer implementation for greater control. This new configuration
element is optional, and defaults mimic prior behavior. The one exception is
that the "root" locale is default. Previously it was the machine's default locale.
(James Dyer)
- SOLR-4084: Add FuzzyLookupFactory, which is like AnalyzingSuggester except that
it can tolerate typos in the input.
(Areek Zillur via Robert Muir)
- SOLR-4088: New and improved auto host detection strategy for SolrCloud.
(Raintung Li via Mark Miller)
- SOLR-3970: SystemInfoHandler now exposes more details about the
JRE/VM/Java version in use.
(hossman)
- SOLR-4101: Add support for storing term offsets in the index via a
'storeOffsetsWithPositions' flag on field definitions in the schema.
(Tom Winch, Alan Woodward)
- SOLR-4093: Solr QParsers may now be directly invoked in the lucene
query syntax without the _query_ magic field hack.
Example: foo AND {!term f=myfield v=$qq}
(yonik)
- SOLR-4087: Add MAX_DOC_FREQ option to MoreLikeThis.
(Andrew Janowczyk via Mark Miller)
- SOLR-4114: Allow creating more than one shard per instance with the
Collection API.
(Per Steffensen, Mark Miller)
- SOLR-3531: Allowing configuring maxMergeSizeMB and maxCachedMB when
using NRTCachingDirectoryFactory.
(Andy Laird via Mark Miller)
- SOLR-4118: Fix replicationFactor to align with industry usage.
replicationFactor now means the total number of copies
of a document stored in the collection (or the total number of
physical indexes for a single logical slice of the collection).
For example if replicationFactor=3 then for a given shard there
will be a total of 3 replicas (one of which will normally be
designated as the leader.)
(yonik)
- SOLR-4124: You should be able to set the update log directory with the
CoreAdmin API the same way as the data directory.
(Mark Miller)
- SOLR-4028: When using ZK chroot, it would be nice if Solr would create the
initial path when it doesn't exist.
(Tomás Fernández Löbbe via Mark Miller)
- SOLR-3948: Calculate/display deleted documents in admin interface.
(Shawn Heisey via Mark Miller)
- SOLR-4030: Allow rate limiting Directory IO based on the IO context.
(Mark Miller, Radim Kolar)
- SOLR-4166: LBHttpSolrServer ignores ResponseParser passed in constructor.
(Steve Molloy via Mark Miller)
- SOLR-4140: Allow access to the collections API through CloudSolrServer
without referencing an existing collection.
(Per Steffensen via Mark Miller)
- SOLR-788: Distributed search support for MLT.
(Matthew Woytowitz, Mike Anderson, Jamie Johnson, Mark Miller)
- SOLR-4120: Collection API: Support for specifying a list of Solr addresses to
spread a new collection across.
(Per Steffensen via Mark Miller)
- SOLR-4110: Configurable Content-Type headers for PHPResponseWriters and
PHPSerializedResponseWriter.
(Dominik Siebel via Mark Miller)
- SOLR-1028: The ability to specify "transient" and "loadOnStartup" as a new properties of
<core> tags in solr.xml. Can specify "transientCacheSize" in the <cores> tag. Together
these allow cores to be loaded only when needed and only transientCacheSize transient
cores will be loaded at a time, the rest aged out on an LRU basis.
- SOLR-4246: When update.distrib is set to skip update processors before
the distributed update processor, always include the log update processor
so forwarded updates will still be logged.
(yonik)
- SOLR-4230: The new Solr 4 spatial fields now work with the {!geofilt} and
{!bbox} query parsers. The score local-param works too.
(David Smiley)
- SOLR-1972: Add extra statistics to RequestHandlers - 5 & 15-minute reqs/sec
rolling averages; median, 75th, 95th, 99th, 99.9th percentile request times
(Alan Woodward, Shawn Heisey, Adrien Grand, Uwe Schindler)
- SOLR-4271: Add support for PostingsHighlighter.
(Robert Muir)
- SOLR-4255: The new Solr 4 spatial fields now have a 'filter' boolean local-param
that can be set to false to not filter. It's useful when there is already a spatial
filter query but you also need to sort or boost by distance.
(David Smiley)
- SOLR-4265, SOLR-4283: Solr now parses request parameters (in URL or sent with POST
using content-type application/x-www-form-urlencoded) in its dispatcher code. It no
longer relies on special configuration settings in Tomcat or other web containers
to enable UTF-8 encoding, which is mandatory for correct Solr behaviour. Query
strings passed in via the URL need to be properly-%-escaped, UTF-8 encoded
bytes, otherwise Solr refuses to handle the request. The maximum length of
x-www-form-urlencoded POST parameters can now be configured through the
requestDispatcher/requestParsers/@formdataUploadLimitInKB setting in
solrconfig.xml (defaults to 2 MiB). Solr now works out of the box with
e.g. Tomcat, JBoss,...
(Uwe Schindler, Dawid Weiss, Alex Rocher)
- SOLR-2201: DIH's "formatDate" function now supports a timezone as an optional
fourth parameter
(James Dyer, Mark Waddle)
- SOLR-4302: New parameter 'indexInfo' (defaults to true) in CoreAdmin STATUS
command can be used to omit index specific information
(Shahar Davidson via shalin)
- SOLR-2592: Collection specific document routing. The "compositeId"
router is the default for collections with hash based routing (i.e. when
numShards=N is specified on collection creation). Documents with ids sharing
the same domain (prefix) will be routed to the same shard, allowing for
efficient querying.
Example:
The following two documents will be indexed to the same shard
since they share the same domain "customerB!".
{"id" : "customerB!doc1" [...] }
{"id" : "customerB!doc2" [...] }
At query time, one can specify a "shard.keys" parameter that lists what
shards the query should cover.
http://.../query?q=my_query&shard.keys=customerB!
Collections that do not specify numShards at collection creation time
use custom sharding and default to the "implicit" router. Document updates
received by a shard will be indexed to that shard, unless a "_shard_" parameter
or document field names a different shard.
(Michael Garski, Dan Rosher, yonik)
- Optimizations (15)
- SOLR-3788: Admin Cores UI should redirect to newly created core details
(steffkes)
- SOLR-3895: XML and XSLT UpdateRequestHandler should not try to resolve
external entities. This improves speed of loading e.g. XSL-transformed
XHTML documents.
(Martin Herfurt, uschindler, hossman)
- SOLR-3614: Fix XML parsing in XPathEntityProcessor to correctly expand
named entities, but ignore external entities.
(uschindler, hossman)
- SOLR-3734: Improve Schema-Browser Handling for CopyField using
dynamicField's
(steffkes)
- SOLR-3941: The "commitOnLeader" part of distributed recovery can use
openSearcher=false.
(Tomás Fernández Löbbe via Mark Miller)
- SOLR-4063: Allow CoreContainer to load multiple SolrCores in parallel rather
than just serially.
(Mark Miller)
- SOLR-4199: When doing zk retries due to connection loss, rather than just
retrying for 2 minutes, retry in proportion to the session timeout.
(Mark Miller)
- SOLR-4262: Replication Icon on Dashboard does not reflect Master-/Slave-
State
(steffkes)
- SOLR-4264: Missing Error-Screen on UI's Cloud-Page
(steffkes)
- SOLR-4261: Percentage Infos on Dashboard have a fixed width
(steffkes)
- SOLR-3851: create a new core/delete an existing core should also update
the main/left list of cores on the admin UI
(steffkes)
- SOLR-3840: XML query response display is unreadable in Solr Admin Query UI
(steffkes)
- SOLR-3982: Admin UI: Various Dataimport Improvements
(steffkes)
- SOLR-4296: Admin UI: Improve Dataimport Auto-Refresh
(steffkes)
- SOLR-3458: Allow multiple Items to stay open on Plugins-Page
(steffkes)
- Bug Fixes (83)
- SOLR-4288: Improve logging for FileDataSource (basePath, relative
resources).
(Dawid Weiss)
- SOLR-4007: Morfologik dictionaries not available in Solr field type
due to class loader lookup problems.
(Lance Norskog, Dawid Weiss)
- SOLR-3560: Handle different types of Exception Messages for Logging UI
(steffkes)
- SOLR-3637: Commit Status at Core-Admin UI is always false
(steffkes)
- SOLR-3917: Partial State on Schema-Browser UI is not defined for Dynamic
Fields & Types
(steffkes)
- SOLR-3939: Consider a sync attempt from leader to replica that fails due
to 404 a success.
(Mark Miller, Joel Bernstein)
- SOLR-3940: Rejoining the leader election incorrectly triggers the code path
for a fresh cluster start rather than fail over.
(Mark Miller)
- SOLR-3961: Fixed error using LimitTokenCountFilterFactory
(Jack Krupansky, hossman)
- SOLR-3933: Distributed commits are not guaranteed to be ordered within a
request.
(Mark Miller)
- SOLR-3939: An empty or just replicated index cannot become the leader of a
shard after a leader goes down.
(Joel Bernstein, yonik, Mark Miller)
- SOLR-3971: A collection that is created with numShards=1 turns into a
numShards=2 collection after starting up a second core and not specifying
numShards.
(Mark Miller)
- SOLR-3988: Fixed SolrTestCaseJ4.adoc(SolrInputDocument) to respect
field and document boosts
(hossman)
- SOLR-3981: Fixed bug that resulted in document boosts being compounded in
<copyField/> destination fields.
(hossman)
- SOLR-3920: Fix server list caching in CloudSolrServer when using more than one
collection list with the same instance.
(Grzegorz Sobczyk, Mark Miller)
- SOLR-3938: prepareCommit command omits commitData causing a failure to trigger
replication to slaves.
(yonik)
- SOLR-3992: QuerySenderListener doesn't populate document cache.
(Shotaro Kamio, yonik)
- SOLR-3995: Recovery may never finish on SolrCore shutdown if the last reference to
a SolrCore is closed by the recovery process.
(Mark Miller)
- SOLR-3998: Atomic update on uniqueKey field itself causes duplicate document.
(Eric Spencer, yonik)
- SOLR-4001: In CachingDirectoryFactory#close, if there are still refs for a
Directory outstanding, we need to wait for them to be released before closing.
(Mark Miller)
- SOLR-4005: If CoreContainer fails to register a created core, it should close it.
(Mark Miller)
- SOLR-4009: OverseerCollectionProcessor is not resilient to many error conditions
and can stop running on errors.
(Raintung Li, milesli, Mark Miller)
- SOLR-4019: Log stack traces for 503/Service Unavailable SolrException if not
thrown by PingRequestHandler. Do not log exceptions if a user tries to view a
hidden file using ShowFileRequestHandler.
(Tomás Fernández Löbbe via James Dyer)
- SOLR-3589: Edismax parser does not honor mm parameter if analyzer splits a token.
(Tom Burton-West, Robert Muir)
- SOLR-4031: Upgrade to Jetty 8.1.7 to fix a bug where in very rare occasions
the content of two concurrent requests get mixed up.
(Per Steffensen, yonik)
- SOLR-4060: ReplicationHandler can try and do a snappull and open a new IndexWriter
after shutdown has already occurred, leaving an IndexWriter that is not closed.
(Mark Miller)
- SOLR-4055: Fix a thread safety issue with the Collections API that could
cause actions to be targeted at the wrong SolrCores.
(Raintung Li, Per Steffensen via Mark Miller)
- SOLR-3993: If multiple SolrCore's for a shard coexist on a node, on cluster
restart, leader election would stall until timeout, waiting to see all of
the replicas come up.
(Mark Miller, Alexey Kudinov)
- SOLR-2045: Databases that require a commit to be issued before closing the
connection on a non-read-only database leak connections. Also expanded the
SqlEntityProcessor test to sometimes use Derby as well as HSQLDB (Derby is
one db affected by this bug).
(Fenlor Sebastia, James Dyer)
- SOLR-4064: When there is an unexpected exception while trying to run the new
leader process, the SolrCore will not correctly rejoin the election.
(Po Rui via Mark Miller)
- SOLR-3989: SolrZkClient constructor dropped exception cause when throwing
a new RuntimeException.
(Colin Bartolome, yonik)
- SOLR-4036: field aliases in fl should not cause properties of target field
to be used.
(Martin Koch, yonik)
- SOLR-4003: The SolrZKClient clean method should not try and clear zk paths
that start with /zookeeper, as this can fail and stop the removal of
further nodes.
(Mark Miller)
- SOLR-4076: SolrQueryParser should run fuzzy terms through
MultiTermAwareComponents to ensure that (for example) a fuzzy query of
foobar~2 is equivalent to FooBar~2 on a field that includes lowercasing.
(yonik)
- SOLR-4081: QueryParsing.toString, used during debugQuery=true, did not
correctly handle ExtendedQueries such as WrappedQuery
(used when cache=false), spatial queries, and frange queries.
(Eirik Lygre, yonik)
- SOLR-3959: Ensure the internal comma separator of poly fields is escaped
for CSVResponseWriter.
(Areek Zillur via Robert Muir)
- SOLR-4075: A logical shard that has had all of its SolrCores unloaded should
be removed from the cluster state.
(Mark Miller, Gilles Comeau)
- SOLR-4034: Check if a collection already exists before trying to create a
new one.
(Po Rui, Mark Miller)
- SOLR-4097: Race can cause NPE in logging line on first cluster state update.
(Mark Miller)
- SOLR-4099: Allow the collection api work queue to make forward progress even
when its watcher is not fired for some reason.
(Raintung Li via Mark Miller)
- SOLR-3960: Fixed a bug where Distributed Grouping ignored PostFilters
(Nathan Visagan, hossman)
- SOLR-3842: DIH would not populate multivalued fields if the column name
derives from a resolved variable
(James Dyer)
- SOLR-4117: Retrieving the size of the index may use the wrong index dir if
you are replicating.
(Mark Miller, Markus Jelsma)
- SOLR-2890: Fixed a bug that prevented omitNorms and omitTermFreqAndPositions
options from being respected in some <fieldType/> declarations
(hossman)
- SOLR-4159: When we are starting a shard from rest, a potential leader should
not consider its last published state when deciding if it can be the new
leader.
(Mark Miller)
- SOLR-4158: When a core is registering in ZooKeeper it may not wait long
enough to find the leader due to how long the potential leader waits to see
replicas.
(Mark Miller, Alain Rogister)
- SOLR-4162: ZkCli usage examples are not correct because the zkhost parameter
is not present and it is mandatory for all commands.
(Tomás Fernández Löbbe via Mark Miller)
- SOLR-4071: Validate that name is pass to Collections API create, and behave the
same way as on startup when collection.configName is not explicitly passed.
(Po Rui, Mark Miller)
- SOLR-4127: Added explicit error message if users attempt Atomic document
updates with either updateLog or DistribUpdateProcessor.
(hossman)
- SOLR-4136: Fix SolrCloud behavior when using "hostContext" containing "_"
or"/" characters. This fix also makes SolrCloud more accepting of
hostContext values with leading/trailing slashes.
(hossman)
- SOLR-4168: Ensure we are using the absolute latest index dir when getting
list of files for replication.
(Mark Miller)
- SOLR-4171: CachingDirectoryFactory should not return any directories after it
has been closed.
(Mark Miller)
- SOLR-4102: Fix UI javascript error if canonical hostname can not be resolved
(steffkes via hossman)
- SOLR-4178: ReplicationHandler should abort any current pulls and wait for
its executor to stop during core close.
(Mark Miller)
- SOLR-3918: Fixed the 'dist-war-excl-slf4j' ant target to exclude all
slf4j jars, so that the resulting war is usable as is provided the servlet
container includes the correct slf4j api and impl jars.
(Shawn Heisey, hossman)
- SOLR-4198: OverseerCollectionProcessor should implement ClosableThread.
(Mark Miller)
- SOLR-4213: Directories that are not shutdown until DirectoryFactory#close
do not have close listeners called on them.
(Mark Miller)
- SOLR-4134: Standard (XML) request writer cannot "set" multiple values into
multivalued field with partial updates.
(Luis Cappa Banda, Will Butler, shalin)
- SOLR-3972: Fix ShowFileRequestHandler to not log a warning in the
(expected) situation of a file not found.
(hossman)
- SOLR-4133: Cannot "set" field to null with partial updates when using the
standard RequestWriter.
(Will Butler, shalin)
- SOLR-4223: "maxFormContentSize" in jetty.xml is not picked up by jetty 8
so set it via solr webapp context file.
(shalin)
- SOLR-4175:SearchComponent chain can't contain two components of the
same class and use debugQuery.
(Tomás Fernández Löbbe via ehatcher)
- SOLR-4244: When coming back from session expiration we should not wait for
the leader to see us in the down state if we are the node that must become
the leader.
(Mark Miller)