# Searching Your Data with Amazon CloudSearch
You specify the terms or values you want to search for with the `q` parameter. How you specify the search criteria depends on which query parser you use. Amazon CloudSearch supports four query parsers:
+ `simple`—search all `text` and `text-array` fields for the specified string. The simple query parser enables you to search for phrases, individual terms, and prefixes. You can designate terms as required or optional, or exclude matches that contain particular terms. To search particular fields, you can specify the fields you want to search with the `q.options` parameter. The `simple` query parser is used by default if the `q.parser` parameter is not specified.
+ `structured`—search specific fields, construct compound queries using Boolean operators, and use advanced features such as term boosting and proximity searching.
+ `lucene`—specify search criteria using the Apache Lucene query parser syntax. If you currently use the Lucene syntax, using the `lucene` query parser enables you to migrate your search services to an Amazon CloudSearch domain without having to completely rewrite your search queries in the Amazon CloudSearch structured search syntax.
+ `dismax`—specify search criteria using the simplified subset of the Apache Lucene query parser syntax defined by the DisMax query parser. If you are currently using the DisMax syntax, using the `dismax` query parser enables you to migrate your search services to an Amazon CloudSearch domain without having to completely rewrite your search queries in the Amazon CloudSearch structured search syntax.
You can use additional search parameters to [control how search results are returned](controlling-search-results.md) and [include additional information](querying-for-more-info.md) such as facets, highlights, and suggestions with your search results.
For information about all of the Amazon CloudSearch search parameters, see the [Search API Reference](search-api.md).
**Topics**
+ [
# Submitting Search Requests to an Amazon CloudSearch Domain
](submitting-search-requests.md)
+ [
# Constructing Compound Queries in Amazon CloudSearch
](searching-compound-queries.md)
+ [text](searching-text.md)
+ [
# Searching for Numbers in Amazon CloudSearch
](searching-numbers.md)
+ [
# Searching for Dates and Times in Amazon CloudSearch
](searching-dates.md)
+ [
# Searching for a Range of Values in Amazon CloudSearch
](searching-ranges.md)
+ [location-based searching and sorting](searching-locations.md)
+ [
# Searching DynamoDB Data with Amazon CloudSearch
](searching-dynamodb-data.md)
+ [
# Filtering Matching Documents in Amazon CloudSearch
](filtering-results.md)
+ [
# Tuning Search Request Performance in Amazon CloudSearch
](tuning-search.md)
# Submitting Search Requests to an Amazon CloudSearch Domain
We recommend using one of the AWS SDKs or the AWS CLI to submit search requests. The SDKs and AWS CLI handle request signing for you and provide an easy way to perform all Amazon CloudSearch actions. You can also use the Search Tester in the Amazon CloudSearch console to search your data, browse the results, and view the generated request URLs and JSON and XML responses. For more information, see [Searching with the Search Tester](getting-started-search.md#searching-console).
**Important**
Search endpoints don't change: A domain's document and search endpoints remain the same for the life of the domain. You should cache the endpoints rather than retrieving them before every upload or search request. Querying the Amazon CloudSearch configuration service by calling `aws cloudsearch describe-domains` or `DescribeDomains` before every request is likely to result in your requests being throttled.
IP addresses **do** change: Your domain's IP address *can* change over time, so it's important to cache the endpoint as shown in the console and returned by the `aws cloudsearch describe-domains` command rather than the IP address. You should also re-resolve the endpoint DNS to an IP address regularly. For more information, see [Setting the JVM TTL for DNS Name Lookups](https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/java-dg-jvm-ttl.html).
For example, the following request submits a simple text search for `wolverine` using the AWS CLI and returns just the IDs of the matching documents.
```
aws cloudsearchdomain --endpoint-url http://search-movies-y6gelr4lv3jeu4rvoelunxsl2e.us-east-1.cloudsearch.amazonaws.com search --search-query wolverine --return _no_fields
{
"status": {
"rid": "/rnE+e4oCAqfEEs=",
"time-ms": 6
},
"hits": {
"found": 3,
"hit": [
{
"id": "tt1430132"
},
{
"id": "tt0458525"
},
{
"id": "tt1877832"
}
],
"start": 0
}
}
```
By default, Amazon CloudSearch returns the response in JSON. You can get the results formatted in XML by specifying the `format` parameter. Setting the response format only affects responses to successful requests. The format of an error response depends on the origin of the error. Errors returned by the search service are always returned in JSON. 5xx errors due to server timeouts and other request routing problems are returned in XML.
**Note**
The AWS SDKs return fields as arrays. Single-value fields are returned as arrays with one element, such as:
```
"fields": {
"plot": ["Katniss Everdeen reluctantly becomes the symbol of a mass rebellion against the autocratic Capitol."]
}
```
For development and testing purposes, you can allow anonymous access to your domain's search service and submit unsigned HTTP GET or POST requests directly to your domain's search endpoint. In a production environment, restrict access to your domain to specific IAM roles, groups, or users and submit signed requests using the AWS SDKs or AWS CLI. For information about controlling access for Amazon CloudSearch, see [Configuring Access for Amazon CloudSearch](configuring-access.md). For more information about request signing, see [Signing AWS API Requests](https://docs.aws.amazon.com/general/latest/gr/signing_aws_api_requests.html).
You can use any method you want to send HTTP requests directly to your domain's search endpoint—you can enter the request URL directly in a Web browser, use cURL to submit the request, or generate an HTTP call using your favorite HTTP library. To specify your search criteria, you specify a query string that specifies the constraints for your search and what you want to get back in the response. The query string must be URL-encoded. The maximum size of a search request submitted via GET is 8190 bytes, including the HTTP method, URI, and protocol version. You can submit larger requests using HTTP POST; however, keep in mind that large, complex requests take longer to process and are more likely to time out. For more information, see [Tuning Search Request Performance in Amazon CloudSearch](tuning-search.md).
For example, the following request submits a structured query to the `search-movies-rr2f34ofg56xneuemujamut52i.us-east-1.cloudsearch.amazonaws.com` domain and gets the contents of the `title` field.
```
http://search-movies-rr2f34ofg56xneuemujamut52i.us-east-1.cloudsearch.
amazonaws.com/2013-01-01/search?q=(and+(term+field%3Dtitle+'star')
(term+field%3Dyear+1977))&q.parser=structured&return=title
```
**Important**
Special characters in the query string must be URL-encoded. For example, you must encode the `=` operator in a structured query as `%3D`: `(term+field%3Dtitle+'star')`. If you don't encode the special characters when you submit the search request, you'll get an `InvalidQueryString` error.
# Searching with the Search Tester
The search tester in the Amazon CloudSearch console enables you to submit sample search requests using any of the supported query parsers: simple, structured, lucene, or dismax. By default, requests are processed with the simple query parser. You can specify options for the selected parser, filter and sort the results, and browse the configured facets. The search hits are automatically highlighted in the search results. For information about how this is done, see [Highlighting Search Hits in Amazon CloudSearch](highlighting.md). You can also select a suggester to get suggestions as you enter terms in the **Search** field. (You must configure a suggester before you can get suggestions. For more information see [Getting Autocomplete Suggestions in Amazon CloudSearch](getting-suggestions.md).)
By default, results are sorted according to an automatically-generated relevance score, * \$1score*. For information about customizing how results are ranked, see [Sorting Results in Amazon CloudSearch](sorting-results.md).
**To search your domain**
1. Go to the Amazon CloudSearch console at [https://console.aws.amazon.com/cloudsearch/home](https://console.aws.amazon.com/cloudsearch/home).
1. In the left navigation panel, choose the name of your domain to open its configuration.
1. Choose **Run a test search**.
1. To perform a simple text search, enter a search query and choose **Run**. By default, all `text` and `text-array` fields are searched.
To search particular fields, expand **Options** and enter a comma-separated list of the fields you want to search in the **Search fields** field. You can append a weight to each field with a caret (^) to control the relative importance of each field in the search results. For example, specifying `title^5, description` weights hits in the `title` field five times more than hits in the `description` field when calculating relevance scores for each matching document.
To use the structured query syntax, select **Structured** from the **Query parser** menu. Once you've selected the structured query parser, enter your structured query in the **Search** field and choose **Run**. For example, to find all of the movies with *star* in the title that were released in the year 2000 or earlier, you could enter: `(and title:'star' year:{,2000])`. For more information, see [Constructing Compound Queries](searching-compound-queries.md). To submit Lucene or DisMax queries, select the appropriate query parser.
You can specify additional options for the selected query parser to configure the default operator and control which operators can be used in a query. For more information, see [Search Request Parameters](search-api.md#search-request-parameters).
You can copy and paste the request URL to submit the request and view the response from a Web browser. Requests can be sent via HTTP or HTTPS.
# Constructing Compound Queries in Amazon CloudSearch
You can use the structured query parser to combine match expressions using Boolean `and`, `or`, and `not` operators. To select the structured query parser, you include `q.parser=structured` in your query. The structured query operators are specified as *prefix* operators. The syntax is:
+ `(and boost=N EXPRESSION1 EXPRESSION2 ... EXPRESSIONn)`
+ `(or boost=N EXPRESSION1 EXPRESSION2 ... EXPRESSIONn)`
+ `(not boost=N EXPRESSION)`
For example, the following query matches all movies in the sample data set that contain *star* in the title, and either Harrison Ford or William Shatner appear in the `actors` field, but Zachary Quinto does not.
```
(and title:'star' (or actors:'Harrison Ford' actors:'William Shatner')(not actors:'Zachary Quinto'))
```
When using the structured query operators, you specify the name of the operator, options for the operator, and then the match expression being operated on, `(OPERATOR OPTIONS EXPRESSION)`. The match expression can be a simple text string, or a subclause of your compound query. Any options must be specified before the terms. For example, `(and (not field=genres 'Sci-Fi')(or (term field=title boost=2 'star')(term field=plot 'star')))`.
Parentheses control the order of evaluation of the expressions. When an expression is enclosed in parentheses, that expression is evaluated first, and then the resulting value is used in the evaluation of the remainder of the compound query.
**Important**
You must URL-encode special characters in the query string. For example, you must encode the `=` operator in a structured query as `%3D`: `(term+field%3Dtitle+'star'`). Amazon CloudSearch returns an `InvalidQueryString` error if special characters are not URL-encoded. For a complete reference of URL-encodings, see the W3C [HTML URL Encoding Reference](http://www.w3schools.com/tags/ref_urlencode.asp).
For example, the following query searches the `title` field for the phrase `star wars` and excludes matches that have a value less than 2000 in the `year` field.
```
(and (phrase field='title' 'star wars') (not (range field=year {,2000})))
```
To submit this search request, you need to encode the query string and specify the `structured` query parser with the `q.parser` parameter.
```
http://search-movies-rr2f34ofg56xneuemujamut52i.us-east-1.cloudsearch.
amazonaws.com/2013-01-01/search?q=(and+(phrase+field='title'+'star wars')+(not+(range+field%3Dyear+{,2000})))&q.parser=structured
```
The structured query syntax enables you to combine searches against multiple fields. If you don't specify a field to search, all `text` and `text-array` fields are searched. For example, the following query searches all `text` and `text-array` fields for the term *star*, and excludes documents that contain *Zachary Quinto* in the `actors` field.
```
(and 'star' (not actors:'Zachary Quinto'))
```
You can specify a `boost` value to increase the importance of one expression in a compound query in relation to the others. The boost value increases the scores of the matching documents. For example, the following query boosts matches for the term *star* if they occur in the `title` field rather than the `description` field.
```
(and (range field=year [2013,}) (or (term field=title boost=2 'star') (term field=plot 'star'))
```
Boost values must be greater than zero.
In addition to `and`, `or`, and `not`, the Amazon CloudSearch structured search syntax supports several specialized operators:
+ `matchall`—Matches every document in the domain. Syntax: `matchall`.
+ `near`—Supports sloppy phrase queries. The `distance` value specifies the maximum number of words that can separate the words in the phrase; for example, `(near field='plot' distance=4 'naval mutiny demonstration')`. Use the `near` operator to enable matching when the specified terms are in close proximity, but not adjacent. For more information about sloppy phrase searches, see [Searching for Phrases](searching-text.md#searching-text-phrases). Syntax: `(near field=FIELD distance=N boost=N 'STRING')`.
+ `phrase`—Searches for a phrase in `text` or `text-array` fields; for example, `(phrase field="title" 'teenage mutant ninja')`. Supports boosting documents that match the expression. For more information about phrase searches, see [Searching for Phrases](searching-text.md#searching-text-phrases). Syntax: `(phrase field=FIELD boost=N 'STRING')`.
+ `prefix`—Searches a text, text-array, literal, or literal-array field for the specified prefix followed by zero or more characters; for example, `(prefix field='title' 'wait')`. Supports boosting documents that match the expression. For more information about prefix searches, see [Searching for Prefixes](searching-text.md#searching-text-prefixes).Syntax: `(prefix field=FIELD boost=N 'STRING')`.
+ `range`—Searches for a range of values in a numeric field; for example: `(range field=year [2000,2013])`. For more information about range searches, see [Searching for a Range of Values](searching-ranges.md). Syntax: `(range field=FIELD boost=N RANGE)`.
+ `term`—Searches for an individual term or value in any field; for example: `(and (term field=title 'star')(term field=year 1977))`. Syntax: `(term field=FIELD boost=N 'STRING'|VALUE)`.
For more information about searching particular types of data, see the following sections. For more information about the structured search syntax, see [Structured Search Syntax](search-api.md#structured-search-syntax).
# Searching for Text in Amazon CloudSearch
You can search both text and literal fields for a text string:
+ `Text` and `text-array` fields are always searchable. You can search for individual terms as well as phrases. Searches within `text` and `text-array` fields are not case-sensitive.
+ `Literal` and `literal-array` fields can only be searched if they are search enabled in the domain's indexing options. You can search for an exact match of your search string. Searches in literal fields are case-sensitive.
If you use the simple query parser or do not specify a field when searching with the structured query parser, by default all `text` and `text-array` fields are searched. Literal fields are *not* searched by default. You can specify which fields you want to search with the `q.options` parameter.
You can search the unique document ID field like any text field. To reference the document ID field in a search request, you use the field name `_id`. Document IDs are always returned in the search results.
**Topics**
+ [
## Searching for Individual Terms in Amazon CloudSearch
](#searching-text-terms)
+ [
## Searching for Phrases in Amazon CloudSearch
](#searching-text-phrases)
+ [
## Searching for Literal Strings in Amazon CloudSearch
](#searching-text-literals)
+ [
## Searching for Prefixes in Amazon CloudSearch
](#searching-text-prefixes)
## Searching for Individual Terms in Amazon CloudSearch
When you search `text` and `text-array` fields for individual terms, Amazon CloudSearch finds all documents that contain the search terms anywhere within the specified field, in any order. For example, in the sample movie data, the `title` field is configured as a `text` field. If you search the `title` field for *star*, you will find all of the movies that contain *star* anywhere in the `title` field, such as *star*, *star wars*, and *a star is born*. This differs from searching `literal` fields, where the field value must be identical to the search string to be considered a match.
The `simple` query parser provides an easy way to search `text` and `text-array` fields for one or more terms. The `simple` query parser is used by default unless you use the `q.parser` parameter to specify a different query parser.
For example, to search for *katniss*, specify `katniss` in the query string. By default, Amazon CloudSearch includes all return enabled fields in the search results. You can specify the `return` parameter to specify which fields you want to return.
```
https://search-domainname-domainid.us-east-1.cloudsearch.amazonaws.com/
2013-01-01/search?q=katniss&return=title
```
By default, the response is returned in JSON:
```
{
"status": {
"rid": "rd+5+r0oMAo6swY=",
"time-ms": 9
},
"hits": {
"found": 3,
"start": 0,
"hit": [
{
"id": "tt1951265",
"fields": {
"title": "The Hunger Games: Mockingjay - Part 1"
}
},
{
"id": "tt1951264",
"fields": {
"title": "The Hunger Games: Catching Fire"
}
},
{
"id": "tt1392170",
"fields": {
"title": "The Hunger Games"
}
}
]
}
}
```
To specify multiple terms, separate the terms with a space. For example: `star wars`. When you specify multiple search terms, by default documents must contain all of the terms to be considered a match. The terms can occur anywhere within the text field, in any order.
By default, all `text` and `text-array` fields are searched when you use the simple query parser. You can specify which fields you want to search by specifying the `q.options` parameter. For example, this query constrains the search to the `title` and `description` fields and boosts the importance of matches in the `title` field over matches in the `description` field.
```
q=star wars&q.options={fields: ['title^5','description']}
```
When you use the simple query parser, you can use the following prefixes to designate individual terms as required, optional, or to be excluded from the search results:
+ **`+`**—matching documents must contain the term. This is the default—separating terms with a space is equivalent to preceding them with the `+` prefix.
+ **`-`**—exclude documents that contain the term from the search results. The `-` operator only applies to individual terms. For example, to exclude documents that contain the term *star* in the default search field, specify: `-star`. Searching for `search?q=-star wars` retrieves all documents that do not contain the term *star*, but do contain the term *wars*.
+ **`|`**—include documents that contain the term in the search results, even if they don't contain the other terms. The `|` operator only applies to individual terms. For example, to include documents that contain either of two terms, specify: `term1 |term2`. Searching for `search?q=star wars |trek` includes documents that contain both *star* and *wars*, or the term *trek*.
These prefixes only apply to individual terms in a simple query. To construct compound queries, you need to use the structured query parser, rather than the simple query parser. For example, to search for the terms *star* and *wars* using the structured query parser you would specify:
```
(and 'star' 'wars')
```
Note that this query matches documents that contain each of the terms in any of the fields being searched. The terms do not have to be in the same field to be considered a match. If, however, you specify `(and 'star wars' 'luke')`, *star* and *wars* must occur within the same field, and *luke* can occur in any of the fields.
If you don't specify any fields when you use the `structured` query parser, all `text` and `text-array` fields are searched by default, just like with the `simple` parser. Similarly, you can use the `q.options` parameter to control which fields are searched and to boost the importance of selected fields. For more information, see [Constructing Compound Queries](searching-compound-queries.md).
You can also perform *fuzzy* searches with the simple query parser. To perform a fuzzy search, append the `~` operator and a value that indicates how much terms can differ from the user query string and still be considered a match. For example, the specifying `planit~1` searches for the term *planit* and allows matches to differ by up to one character, which means the results will include hits for *planet*.
## Searching for Phrases in Amazon CloudSearch
When you search for a phrase, Amazon CloudSearch finds all documents that contain the complete phrase in the order specified. You can also perform *sloppy* phrase searches where the terms appear within the specified distance of one another.
To match a complete phrase rather than the individual terms in the phrase when you search with the simple query parser, enclose the phrase in double quotes. For example, the following query searches for the phrase *with love*.
```
q="with love"
```
To perform a sloppy phrase search with the simple query parser, append the `~` operator and a distance value. The distance value specifies the maximum number of words that can separate the words in the phrase. For example, the following query searches for the terms *with love* within three words of one another.
```
q="with love"~3
```
In a compound query, you use the `phrase` operator to specify the phrase you want to match; for example:
```
(phrase field=title 'star wars')
```
To perform a sloppy phrase search in a compound query, you use the `near` operator. The `near` operator enables you to specify the phrase you are looking for and how far apart the terms can be within a field and still be considered a match. For example, the following query matches documents that have the terms *star* and *wars* no more than three words apart in the `title` field.
```
(near field=title distance=3 'star wars')
```
For more information, see [Constructing Compound Queries](searching-compound-queries.md).
## Searching for Literal Strings in Amazon CloudSearch
When you search a literal field for a string, Amazon CloudSearch returns only those documents that contain an exact match for the complete search string in the specified field, including case. For example, if the `title` field is configured as a literal field and you search for *Star*, the value of the `title` field must be *Star* to be considered a match—*star*, *star wars* and *a star is born* will not be included in the search results. This differs from text fields, where searches are not case-sensitive and the specified search terms can appear anywhere within the field in any order.
To search a literal field, prefix the search string with the name of the literal field you want to search, followed by a colon. The search string must be enclosed in single quotes. For example, the following query searches for the literal string *Sci-Fi*.
```
genres:'Sci-Fi'
```
This example searches the genre field of each document and matches all documents whose genre field contains the value *Sci-Fi*. To be a match, the field value must be an exact match for the search string, including case. For example, documents that contain the value *Sci-Fi* in the genre field will not be included in the search results if you search for *sci-fi* or *young adult sci-fi*.
In a compound query, you use the `term` operator syntax to search literal fields. For example, `(term field=genres 'Sci-Fi')`. For more information, see [Constructing Compound Queries](searching-compound-queries.md).
You can use literal fields in conjunction with faceting to enable users to drill down into the results according to the faceted attributes. For more information about faceting, see [Getting and Using Facet Information in Amazon CloudSearch](faceting.md).
## Searching for Prefixes in Amazon CloudSearch
You can search `text`, `text-array`, `literal`, and `literal-array` fields for a *prefix* rather than for a complete term. This matches results that contain the prefix followed by zero or more characters. You must specify at least one character as the prefix. (To match all documents, use the `matchall` operator in a structured query.) In general, you should use a prefix that contains at least two characters to avoid matching an excessive number of documents.
When you search a `text` or `text-array` field, terms that match the prefix can occur anywhere within the contents of the field. When you search literal fields, the entire search string, up to and including the prefix characters, must match exactly.
+ Simple query parser—use the `*` (asterisk) wildcard operator to search for a prefix, for example `pre*`.
+ Structured query parser—use the `prefix` operator to search for a prefix, for example `prefix 'pre'`
For example, the following query searches for the prefix *oce* in the title field and returns the title of each hit:
```
q=oce*&q.options={fields:['title']}&return=title
```
If you perform this search against the sample movie data, it returns as *Ocean's Eleven* and *Ocean's Twelve*:
```
{
"status": {
"rid": "hIbIxb8oRAo6swY=",
"time-ms": 2
},
"hits": {
"found": 2,
"start": 0,
"hit": [
{
"id": "tt0240772",
"fields": {
"title": "Ocean's Eleven"
}
},
{
"id": "tt0349903",
"fields": {
"title": "Ocean's Twelve"
}
}
]
}
}
```
In a compound query, you use the `prefix` operator to search for prefixes. For example, to search the `title` field for the prefix *oce*, you specify:
```
q.parser=structured&q=(prefix field%3Dtitle 'oce')
```
Note the URL encoding. For more information, see [Constructing Compound Queries](searching-compound-queries.md).
**Note**
When performing wildcard searches on text fields, keep in mind that Amazon CloudSearch tokenizes the text fields during indexing and performs stemming according to the analysis scheme configured for the field. Normally, Amazon CloudSearch performs the same text processing on the search query. However, when you search for a prefix with the wildcard operator (\$1) or `prefix` operator, no stemming is performed on the prefix. This means that a search for a prefix that ends in `s` won't match the singular version of the term. This can happen for any term that ends in `s`, not just plurals. For example, if you search the `actor` field in the sample movie data for `Anders`, there are three matching movies. If you search for `Ander*`, you get those movies as well as several others. However, if you search for `Anders*` there are no matches. This is because the term is stored in the index as `ander`, `anders` does not appear in the index. For more information about how Amazon CloudSearch processes text and how it can affect searches, see [Text Processing in Amazon CloudSearch](text-processing.md).
# Searching for Numbers in Amazon CloudSearch
You can use structured queries to search any search enabled numeric field for a particular value or [range of values](searching-ranges.md). Amazon CloudSearch supports four numeric field types: `double`, `double-array`, `int`, and `int-array`. For more information, see [Configuring Index Fields](configuring-index-fields.md).
The basic syntax for searching a field for a single value is **FIELD**:**VALUE**. For example, `year:2010` searches the sample movie data for movies released in 2010.
You must use the structured query parser to use the field syntax. Note that numeric values are *not* enclosed in quotes—quotes designate a value as a string. To search for a range of values, use a comma (,) to separate the upper and lower bounds, and enclose the range using brackets or braces. For more information, see [Searching for a Range of Values](searching-ranges.md).
In a compound query, you use the `term` operator syntax to search for a single value: `(term field=year 2010)`.
# Searching for Dates and Times in Amazon CloudSearch
You can use structured queries to search any search enabled date field for a particular date and time or a [date-time range](searching-ranges.md). Amazon CloudSearch supports two date field types, `date` and `date-array`. For more information, see [Configuring Index Fields](configuring-index-fields.md).
Dates and times are specified in UTC (Coordinated Universal Time) according to [IETF RFC3339](http://tools.ietf.org/html/rfc3339): `yyyy-mm-ddTHH:mm:ss.SSSZ`. In UTC, for example, 5:00 PM August 23, 1970 is: `1970-08-23T17:00:00Z`. Note that you can also specify fractional seconds when specifying times in UTC. For example, `1967-01-31T23:20:50.650Z.`
To search for a date (or time) in a `date` field, you must enclose the date string in single quotes. For example, both of the following queries search the movie data for all movies released at midnight on December 25, 2001:
```
q.parser=structured&q=release_date:'2001-12-25T00:00:00Z'
q.parser=structured&q=(term field%3Drelease_date '2001-12-25T00:00:00Z')
```
To search an entire day, see [Searching for a Date Range](searching-ranges.md#searching-ranges-dates).
# Searching for a Range of Values in Amazon CloudSearch
You can use structured queries to search a field for a range of values. To specify a range of values, use a comma (,) to separate the upper and lower bounds and enclose the range using brackets or braces. A square brace, [ or ], indicates that the bound is included in the range, a curly brace, \$1 or \$1, excludes the bound.
For example, to search the sample data set for movies released from 2008 to 2010 (inclusive), specify the range as `[2008,2010]`.
To specify an open-ended range, omit the bound. For example, `year:[2002,}` matches all movies released from 2002 onward, and `year:{,1970]` matches all movies released through 1970. When you omit a bound, you must use a curly brace.
In a compound query, you use the `range` operator syntax to search for a range of values; for example: `(range field=year [1967,})`.
## Searching for a Date Range
To search for a range of dates (or times) in a `date` field, you use the same bracketed range syntax that you use for numeric values, but you must enclose the date string in single quotes. For example, the following request searches the movie data for all movies with a release date of January 1, 2013 or later:
```
q.parser=structured&q=release_date:['2013-01-01T00:00:00Z',}
```
Use the following syntax to search for a fixed range:
```
q.parser=structured&q=release_date:['2013-01-01T00:00:00Z','2013-01-02T23:59:59Z']
```
## Searching for a Location Range
You can perform a bounding box search by searching for a range of locations. To search for a range of locations in a `latlon` field, you use the same bracketed range syntax that you use for numeric values, but you must enclose the latitude/longitude pair in single quotes.
For example, if you include a `location` field in each document, you could specify your bounding box filter as `location:['nn.n,nn.n','nn.n,nn.n']`. In the following example, the matches for *restaurant* are filtered so that only matches within the downtown area of Paso Robles, CA are included in the results.
```
q='restaurant'&fq=location:['35.628611,-120.694152','35.621966,-120.686706']&q.parser=structured
```
For more information, see [Searching and Ranking Results by Geographic Location in Amazon CloudSearch](searching-locations.md).
## Searching for a Text Range
You can also search a text or literal field for a range of values using the bracketed range syntax. Like dates, the text strings must be enclosed in single quotes. For example, the following request searches the movie data for a range of document IDs. To reference a document's ID, you use the special field name `_id`.
```
_id:['tt1000000','tt1005000']
```
# Searching and Ranking Results by Geographic Location in Amazon CloudSearch
If you store locations in your document data using a `latlon` field, you can use the `haversin` function in an Amazon CloudSearch expression to compute the distance between two locations. Storing locations with your document data also enables you to easily search within particular areas.
**Topics**
+ [
## Searching Within an Area in Amazon CloudSearch
](#within-area)
+ [
## Sorting Results by Distance in Amazon CloudSearch
](#sorting-by-distance)
## Searching Within an Area in Amazon CloudSearch
To associate a location with a search document, you can store the location's latitude and longitude in a `latlon` field using decimal degree notation. The values are specified as a comma-separated list, `lat,lon`—for example `35.628611,-120.694152`. Associating a location with a document enables you to easily constrain search hits to a particular area with the `fq` parameter.
**To use a bounding box to constrain results to a particular area**
1. Determine the latitude and longitude of the upper-left and lower-right corners of the area you are interested in.
1. Use the `fq` parameter to filter the matching documents using those bounding box coordinates. For example, if you include a `location` field in each document, you could specify your bounding box filter as `fq=location:['nn.n,nn.n','nn.n,nn.n'] `. In the following example, the matches for *restaurant* are filtered so that only matches within the downtown area of Paso Robles, CA are included in the results.
```
q='restaurant'&fq=location:['35.628611,-120.694152','35.621966,-120.686706']&q.parser=structured
```
## Sorting Results by Distance in Amazon CloudSearch
You can define an expression as part of your search request to sort results by distance. Amazon CloudSearch expressions support the `haversin` function, which computes the great-circle distance between two points on a sphere using the latitude and longitude of each point. (For more information, see [Haversine formula](http://en.wikipedia.org/wiki/Haversine_formula).) The resulting distance is returned in kilometers.
To calculate the distance between each matching document and the user, you pass the user's location into the `haversin` function and reference the document locations stored in a `latlon` field. You specify the user latitude and longitude in decimal degree notation and access the latitude and longitude stored in a `latlon` as `FIELD.latitude` and `FIELD.longitude`. For example, `expr.distance=haversin(userlat,userlon, location.latitude,location.longitude)`.
To use the expression to sort the search results, you specify the `sort` parameter.
For example, the following query searches for restaurants and sorts the results by distance from the user.
```
q=restaurant&expr.distance=haversin(35.621966,-120.686706,location.latitude,location.longitude)&sort=distance asc
```
Note that you must explicitly specify the sort direction, `asc` or `desc`.
You can include the distance calculated for each document in the search results by specifying the name of the expression with the `return` parameter. For example, `return=distance`.
You can also use the distance value in more complex expressions to take other characteristics into account, such as a document's relevance `_score`. In the following example, a second rank expression uses both the document's calculated `distance` and its relevance `_score`.
```
expr.distance=haversin(38.958687,-77.343149,latitude,longitude)&expr.myrank=_score/log10(distance)&sort=myrank+desc
```
**Tip**
For these sample queries to work, you must [configure your index](configuring-index-fields.md) with a `latlon` field and have `location` data in your documents:
```
{
"fields": {
"location": "40.05830,-74.40570"
}
}
```
If the field doesn't exist, you might receive the following error message when performing a search:
```
Syntax error in query: field (location) does not exist.
```
For more information about using expressions to sort search results, see [Controlling Search Results](controlling-search-results.md).
# Searching DynamoDB Data with Amazon CloudSearch
You can specify a DynamoDB table as a source when configuring indexing options or uploading data to a search domain through the console. This enables you to quickly set up a search domain to experiment with searching data stored in DynamoDB database tables.
To keep your search domain in sync with changes to the table, you can send updates to both your table and your search domain, or you can periodically load the entire table into a new search domain.
**Topics**
+ [
## Configuring an Amazon CloudSearch Domain to Search DynamoDB Data
](#searching-dynamodb-data.configuring)
+ [
## Uploading Data to Amazon CloudSearch from DynamoDB
](#searching-dynamodb-data.uploading)
+ [
## Synchronizing a Search Domain with a DynamoDB Table
](#searching-dynamodb-data.sync)
## Configuring an Amazon CloudSearch Domain to Search DynamoDB Data
The easiest way to configure a search domain to search DynamoDB data is to use the Amazon CloudSearch console. The console's configuration wizard analyzes your table data and suggests indexing options based on the attributes in the table. You can modify the suggested configuration to control which table attributes are indexed.
**Note**
To upload data from DynamoDB, you must have permission to access both the service and the resources you want to upload. For more information, see [Using IAM to Control Access to DynamoDB Resources](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/UsingIAMWithDDB.html).
When you automatically configure a search domain from a DynamoDB table, a maximum of 200 unique attributes can be mapped to index fields. (You cannot configure more than 200 fields for a search domain, so you can only upload data from DynamoDB tables with 200 or fewer attributes.) When Amazon CloudSearch detects an attribute that has a small number of distinct values, the field is facet enabled in the suggested configuration.
**Important**
When you use a DynamoDB table to configure a domain, the data is not automatically uploaded to the domain for indexing. You must upload the data for indexing as a separate step after you configure the domain.
### Configuring a Domain to Search DynamoDB using the Amazon CloudSearch Console
You can use the Amazon CloudSearch console to analyze data from a DynamoDB table to configure a search domain. A maximum of 5 MB is read from the table regardless of the table size. By default, Amazon CloudSearch reads from the beginning of the table. You can specify a start key to begin reading from a particular item.
**To configure a search domain using a DynamoDB table**
1. Open the Amazon CloudSearch console at [https://console.aws.amazon.com/cloudsearch/home](https://console.aws.amazon.com/cloudsearch/home).
1. From the left navigation pane, choose **Domains**.
1. Choose the name of the domain to open its details panel.
1. Go to the **Indexing options** tab and choose **Configuration wizard**.
1. Select **Amazon DynamoDB**.
1. Select the DynamoDB table that you want to analyze.
+ To limit the read capacity units that can be consumed while reading from the table, enter the maximum percentage of read capacity units you want to use.
+ To start reading from a particular item, specify a **Start hash key**. If the table uses a hash and range type primary key, specify both the hash attribute and the range attribute for the item.
1. Choose **Next**.
1. Review the suggested configuration. You can edit these fields and add additional fields.
1. When you finish, choose **Confirm**.
1. If you haven't uploaded data to your domain yet, clear the **Run indexing now** checkbox to exit without indexing. If you're done making configuration changes and are ready to index your data with the new configuration, make sure **Run indexing now** is selected. When you're ready to apply the changes, choose **Finish**.
## Uploading Data to Amazon CloudSearch from DynamoDB
You can upload DynamoDB data to a search domain through the Amazon CloudSearch console or with the Amazon CloudSearch command line tools. When you upload data from a DynamoDB table, Amazon CloudSearch converts it to document batches so it can be indexed. You select define index fields for each of the attributes in your domain configuration. For more information, see [Configuring an Amazon CloudSearch Domain to Search DynamoDB Data](#searching-dynamodb-data.configuring).
You can upload data from more than one DynamoDB table to the same Amazon CloudSearch domain. However, keep in mind that you can upload a maximum of 200 attributes from all tables combined. If an item with the same key appears in more than one uploaded table, the last-applied item overwrites all previous versions.
When converting table data to document batches, Amazon CloudSearch generates a document for each item it reads from the table, and represents each item attribute as a document field. The unique ID for each document is either read from the `docid` item attribute (if it exists) or assigned an alphanumeric value based on the primary key.
When Amazon CloudSearch generates documents for table items:
+ Sets of strings and sets of numbers are represented as multi-value fields. If a DynamoDB set contains more than 100 values, only the first 100 values are added to the multi-value field.
+ DynamoDB binary attributes are ignored.
+ Attribute names are modified to conform to the Amazon CloudSearch naming conventions for field names:
+ All uppercase letters are converted to lowercase.
+ If the DynamoDB attribute name does not begin with a letter, the field name is prefixed with `f_`.
+ Any characters other than a-z, 0-9, and \$1 (underscore) are replaced by an underscore. If this transformation results in a duplicate field name, a number is appended to make the field name unique. For example, the attribute names `håt`, `h-t`, `hát` would be mapped to `h_t`, `h_t1`, and `h_t2` respectively.
+ If the DynamoDB attribute name exceeds 64 characters, the first 56 characters of the attribute name are concatenated with the 8-character MD5 hash of the full attribute name to form the field name.
+ If the attribute name is `body`, it is mapped to the field name `f_body`.
+ If the attribute name is ` _score` it is mapped to the field name ` f_ _score`.
+ Number attributes are mapped to Amazon CloudSearch int fields and the values are transformed to 32-bit unsigned integers:
+ If a number attribute contains a decimal value, only the integral part of the value is stored. Everything to the right of the decimal point is dropped.
+ If the value is larger than can be stored as an unsigned integer, the value is truncated.
+ Negative integers are treated as unsigned positive integers.
### Uploading DynamoDB Data to a Domain through the Amazon CloudSearch Console
You can use the Amazon CloudSearch console to upload up to 5 MB of data from a DynamoDB table to a search domain.
**To upload DynamoDB data using the console**
1. Open the Amazon CloudSearch console at [https://console.aws.amazon.com/cloudsearch/home](https://console.aws.amazon.com/cloudsearch/home).
1. From the left navigation pane, choose **Domains**.
1. Choose the name of the domain to open its configuration.
1. Choose **Actions**, **Upload documents**.
1. Select **Amazon DynamoDB**.
1. From the dropdown, select the DynamoDB table that contains your data.
+ To limit the read capacity units that can be consumed while reading from the table, enter the maximum percentage of read capacity units.
+ To start reading from a particular item, specify a **Start hash key**. If the table uses a hash and range type primary key, specify both the hash attribute and the range attribute for the item.
1. When you finish specifying the table options, choose **Next**.
1. Review the items that will be uploaded. You can also save the generated document batch by choosing **Download the generated document batch**. Then choose **Upload documents**.
## Synchronizing a Search Domain with a DynamoDB Table
To keep your search domain in sync with updates to your DynamoDB table, you can either programmatically track and apply updates to your domain, or periodically create a new domain and upload the entire table again. If you have a large amount of data, it's best to track and apply updates programmatically.
### Programmatically Synchronizing Updates
To synchronize changes and additions to your DynamoDB table, you can create a separate update table to track the changes to the table you are searching and periodically upload the contents of the update table to the corresponding search domain.
To remove documents from the search domain, you must generate and upload document batches that contain a delete operation for each deleted document. One option is to use a separate DynamoDB table to track deleted items, periodically process the table to generate a batch of delete operations, and upload the batch to your search domain.
To make sure that you don't lose any changes that are made during the initial data upload, you must begin collecting tracking changes before the initial data upload. While you might update some Amazon CloudSearch documents with identical data, you ensure that no changes are lost and your search domain contains an up-to-date version of every document.
How often you synchronize updates depends on the volume of changes and your update latency tolerance. One approach is to accumulate changes over a fixed time period and at the end of the time period upload the changes and delete the period's tracking tables.
For example, to synchronize changes and additions once a day, at the beginning of each day you could create a table called updates\$1YYYY\$1MM\$1DD to collect the daily updates. At the end of the day, you upload the updates\$1YYYY\$1MM\$1DD table to your search domain. After the upload is complete, you can delete the update table and create a new one for the next day.
### Switching to a New Search Domain
If you don't want to track and apply individual updates to your table, you can periodically load the entire table into a new search domain and then switch your query traffic over to the new domain.
**To switch to a new search domain**
1. Create a new search domain and copy the configuration from your existing domain.
1. Upload the entire DynamoDB table to the new domain. For more information, see [Uploading Data to Amazon CloudSearch from DynamoDB](#searching-dynamodb-data.uploading).
1. After the new domain is active, update the DNS entry that directs query traffic to the old search domain to point to the new domain. For example, if you use [Amazon Route 53](http://aws.amazon.com/route53/), you can simply update the recordset with your new search service endpoint.
1. Delete the old domain.
# Filtering Matching Documents in Amazon CloudSearch
You use the `fq` parameter to filter the documents that match the search criteria specified with the `q` parameter without affecting the relevance scores of the documents included in the search results. Specifying a filter just controls which matching documents are included in the results, it has no effect on how they are scored and sorted.
The `fq` parameter supports the structured query syntax described in [Search API Reference](search-api.md).
For example, you could add an `available` field to your documents to indicate whether or not an item is in stock, and filter on that field to limit the results to in-stock items:
```
search?q=star+wars&fq=available:'true'&return=title
```
# Tuning Search Request Performance in Amazon CloudSearch
Search requests can become very resource intensive to process, which can have an impact on the performance and cost of running your search domain. In general, searches that return a large volume of hits and complex structured queries are more resource intensive than simple text queries that match a small percentage of the documents in your search domain.
If you experience slow response times, frequently encounter internal server errors (typically 507 or 509 errors), or see the number of instance hours your search domain consumes increase without a substantial increase in the volume of data you're searching, you can fine-tune your search requests to help reduce the processing overhead. This section reviews what to look for and steps you can take to tune your search requests.
## Analyzing Query Latency
Before you can tune your requests, you must analyze your current search performance. Log your search requests and response times so that you can see which requests take the longest to process. Slow searches can disproportionally affect overall performance by tying up your search domain's resources. Optimizing the slowest search requests speeds up *all* of your searches.
**Topics**
+ [
### Reducing the Number of Hits
](#tuning-search-numdocs)
+ [
### Simplifying Structured Queries
](#simplifying-structured-queries)
### Reducing the Number of Hits
Query latency is directly proportional to the number of matching documents. Searches that match the most documents are generally the slowest.
Eliminating two types of searches that commonly result in a huge number of matching documents can significantly improve overall performance:
+ Queries that match every document in your corpus (`matchall`). While this can be a convenient way to list all the documents in your domain, it's a resource intensive query. If you have a lot of documents, not only can it cause other requests to time out, it's likely to time out itself.
+ Prefix (wildcard) searches with only one or two characters specified. If you're using this type of search to provide instant results as the user types, wait until the user has entered at least two characters before you start submitting requests and displaying the possible matches.
To reduce the number of documents that match your requests, you can also do the following:
+ Eliminate irrelevant words from your corpus so they aren't using for matching. The easiest way to do this is to add them to the stopwords list dictionary for the analysis scheme(s) you're using. Alternatively, you can preprocess your data to strip out irrelevant words. Eliminating irrelevant words also has the benefit of reducing the size of your index, which can help reduce costs.
+ Explicitly filter the results based on the value of a particular field using the `fq` parameter.
If you still have requests that match a lot of documents, you can reduce latency by minimizing the amount of processing to be done on the result set:
+ Minimize the facet information that you request. Generating the facet counts adds to the time it takes to process the request and increases the likelihood that other requests will time out. If you do request facet information, keep in mind that the more facets you specify, the longer it takes to process the request.
+ Avoid using your own expressions for sorting. The additional processing required to sort the results increases the likelihood that requests will time out. If you must customize how the results are sorted, it is generally faster to use a field than to use an expression.
Keep in mind that returning a large amount of data in the search results can increase the transport time and affect query latency. Minimize the number of return fields you use to improve performance and reduce the size of your index.
### Simplifying Structured Queries
The more clauses there are in the query criteria, the longer it takes to process the query.
If you have complex structured queries that don't perform well, you need to find a way to reduce the number of clauses. In some cases, you might simply be able to set a limit or reformulate the query. In others, you might need to modify your domain configuration to accommodate simpler queries.