elastic/elasticsearch

geo type context suggester doesn't allow path to refer to stored field

Open

#28,669 opened on Feb 13, 2018

View on GitHub
 (4 comments) (1 reaction) (0 assignees)Java (25,882 forks)batch import
:Search Relevance/Suggesters>bugTeam:Search Relevancehelp wantedpriority:normal

Repository metrics

Stars
 (76,700 stars)
PR merge metrics
 (Avg merge 2d) (1,000 merged PRs in 30d)

Description

Elasticsearch version (bin/elasticsearch --version): Version: 5.4.2

JVM version (java -version): 1.8.0_141

OS version (uname -a if on a Unix-like system): Linux 3.10.0-693.11.6.el7.x86_64 #1 SMP Thu Jan 4 01:06:37 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Description of the problem including expected versus actual behavior: When path field in geo type context suggester refers to a stored field, ES fails at indexing documents with error:

{
  "error": {
    "root_cause": [
      {
        "type": "number_format_exception",
        "reason": "For input string: \"43.6624803, -79.3863353\""
      }
    ],
    "type": "number_format_exception",
    "reason": "For input string: \"43.6624803, -79.3863353\""
  },
  "status": 400
}

Steps to reproduce:

  1. Create a new index with following mapping:
PUT place2
{
    "mappings": {
        "shops" : {
            "properties" : {
                "suggest" : {
                    "type" : "completion",
                    "contexts": [
                        { 
                            "name": "place_type",
                            "type": "category"
                        },
                        { 
                            "name": "location",
                            "type": "geo",
                            "precision": 4,
                            "path": "loc"
                        }
                    ]
                },
                "loc": {
                  "type": "geo_point",
                  "store": true
                }
            }
        }
    }
}
  1. Index a document with loc field filled in:
PUT place2/shops/2
{
    "suggest": {
        "input": "timmy's",
        "contexts": {
            "place_type": ["cafe", "food"]
        }
    },
    "loc": {
      "lat": 43.6624803,
      "lon": -79.3863353
    }
}

Provide logs (if relevant):

[2018-02-13T15:02:28,521][DEBUG][o.e.a.b.TransportShardBulkAction] [anmao] [place2][2] failed to execute bulk item (index) BulkShardRequest [[place2][2]] containing [index {[place2][shops][2], source[{
    "suggest": {
        "input": "timmy's",
        "contexts": {
            "place_type": ["cafe", "food"]
        }
    },
    "loc": {
      "lat": 43.6624803,
      "lon": -79.3863353
    }
}
]}]
java.lang.NumberFormatException: For input string: "43.6624803, -79.3863353"
        at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) ~[?:1.8.0_141]
        at java.lang.Long.parseLong(Long.java:589) ~[?:1.8.0_141]
        at java.lang.Long.parseLong(Long.java:631) ~[?:1.8.0_141]
        at org.elasticsearch.common.geo.GeoPoint.resetFromIndexableField(GeoPoint.java:113) ~[elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.search.suggest.completion.context.GeoContextMapping.parseContext(GeoContextMapping.java:211) ~[elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.search.suggest.completion.context.ContextMappings$TypedContextField.contexts(ContextMappings.java:134) ~[elasticsearch-5.4.2.jar:5.4.2]
        at org.apache.lucene.search.suggest.document.ContextSuggestField.wrapTokenStream(ContextSuggestField.java:87) ~[lucene-suggest-6.5.1.jar:6.5.1 cd1f23c63abe03ae650c75ec8ccb37762806cc75 - jimczi - 2017-04-21 12:18:13]
        at org.apache.lucene.search.suggest.document.SuggestField.tokenStream(SuggestField.java:103) ~[lucene-suggest-6.5.1.jar:6.5.1 cd1f23c63abe03ae650c75ec8ccb37762806cc75 - jimczi - 2017-04-21 12:18:13]
        at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:740) ~[lucene-core-6.5.1.jar:6.5.1 cd1f23c63abe03ae650c75ec8ccb37762806cc75 - jimczi - 2017-04-21 12:17:15]
        at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:447) ~[lucene-core-6.5.1.jar:6.5.1 cd1f23c63abe03ae650c75ec8ccb37762806cc75 - jimczi - 2017-04-21 12:17:15]
        at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:403) ~[lucene-core-6.5.1.jar:6.5.1 cd1f23c63abe03ae650c75ec8ccb37762806cc75 - jimczi - 2017-04-21 12:17:15]
        at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:232) ~[lucene-core-6.5.1.jar:6.5.1 cd1f23c63abe03ae650c75ec8ccb37762806cc75 - jimczi - 2017-04-21 12:17:15]
        at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:478) ~[lucene-core-6.5.1.jar:6.5.1 cd1f23c63abe03ae650c75ec8ccb37762806cc75 - jimczi - 2017-04-21 12:17:15]
        at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1569) ~[lucene-core-6.5.1.jar:6.5.1 cd1f23c63abe03ae650c75ec8ccb37762806cc75 - jimczi - 2017-04-21 12:17:15]
        at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1314) ~[lucene-core-6.5.1.jar:6.5.1 cd1f23c63abe03ae650c75ec8ccb37762806cc75 - jimczi - 2017-04-21 12:17:15]
        at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:639) ~[elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.index.engine.InternalEngine.indexIntoLucene(InternalEngine.java:583) ~[elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:483) ~[elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:560) ~[elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:549) ~[elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.action.bulk.TransportShardBulkAction.executeIndexRequestOnPrimary(TransportShardBulkAction.java:484) ~[elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.action.bulk.TransportShardBulkAction.executeBulkItemRequest(TransportShardBulkAction.java:143) [elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:113) [elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:69) [elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:939) [elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:908) [elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.action.support.replication.ReplicationOperation.execute(ReplicationOperation.java:113) [elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.onResponse(TransportReplicationAction.java:322) [elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.onResponse(TransportReplicationAction.java:264) [elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.action.support.replication.TransportReplicationAction$1.onResponse(TransportReplicationAction.java:888) [elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.action.support.replication.TransportReplicationAction$1.onResponse(TransportReplicationAction.java:885) [elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.index.shard.IndexShardOperationsLock.acquire(IndexShardOperationsLock.java:147) [elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.index.shard.IndexShard.acquirePrimaryOperationLock(IndexShard.java:1656) [elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.action.support.replication.TransportReplicationAction.acquirePrimaryShardReference(TransportReplicationAction.java:897) [elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.action.support.replication.TransportReplicationAction.access$400(TransportReplicationAction.java:93) [elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.doRun(TransportReplicationAction.java:281) [elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:260) [elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:252) [elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) [elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.transport.TransportService$7.doRun(TransportService.java:627) [elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:638) [elasticsearch-5.4.2.jar:5.4.2]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.4.2.jar:5.4.2]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_141]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_141]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_141]

Contributor guide

Research direction
Examine the parsing logic in GeoPoint.resetFromIndexableField and GeoContextMapping.parseContext to understand why a stored geo point field (lat/lon object) is being read as a string and causes a NumberFormatException.
Tech stack
javaelasticsearch
Domain
backendsearchdatabase
Issue type
Bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
JavaElasticsearch internals
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
40