elastic/elasticsearch

Allow specifying index names in request body

Open

#61,141 opened on Aug 14, 2020

View on GitHub
 (6 comments) (1 reaction) (0 assignees)Java (25,882 forks)batch import
:Core/Infra/REST API>enhancementTeam:Core/Infrahelp wanted

Repository metrics

Stars
 (76,700 stars)
PR merge metrics
 (Avg merge 2d) (1,000 merged PRs in 30d)

Description

Many ES APIs currently provide a way to query for a set of indices by specifying comma separated index patterns. In certain scenarios, scripted querying of these endpoints can create very long URIs (+4096 bytes) that trigger an ES exception (https://github.com/elastic/kibana/issues/72616, https://discuss.elastic.co/t/url-too-long-too-many-indices/143593/4) or could run into a web server's URI length limit in certain configurations (although these are typically longer).

This is constrained to scenarios where:

  1. You want a very specific subset of indices on the fly (aliases, index patterns or data streams probably cannot help)
  2. The subset are many such that the names CSV'd (probably URI encoded too) are more than 4096 bytes
  3. You are probably interacting with ES in scripted programmatic way to achieve specific functionality (i.e., this is a latent issue that will be triggered once the number of indices has grown large)

It seems the index names would need to passed through in some other way (perhaps a PUT or GET with a body?) or creating and deleting aliases on the fly although I am not sure that would have been viable in a case like https://github.com/elastic/kibana/issues/72616, but fortunately the new resolve endpoint could be used to get the information that was needed.

Is there a general way of setting things up so that this limit will not be hit given the current APIs?

Contributor guide