Batch Data Queries#

Batch data requests are an API feature for querying multiple keys in multiple datasets. Responses include records for the keys from each dataset. This is convenient way to get data you want for multiple entities.

Format:

GET /data/WORKSPACE/DATASET_A,DATASET_B/KEY_1,KEY_2

The following table explains how to fill in the request.

Placeholder

Replace with…

WORKSPACE

Workspace is CORE for IEX Cloud Data or the workspace name for your datasets. A batch data request can work in only one workspace at a time. You can optionally create a view from IEX Cloud Financial Data and your workspace data, and batch query that view.

DATASET_A,DATASET_B

Example: QUOTE,NEWS. The list of datasets to query. Note, each Dataset ID is specified in the dataset’s reference page. For example, see the Cash Flow reference page.

KEY_1,KEY_2

Example: AAPL,MSFT,TSLA. The list of symbols to query for in the datasets. A dataset’s primary index is labeled key in the dataset reference page’s Path Parameters and Response Attributes sections.

batchSeparator=,

(Optional) A character for separating the request’s dataset IDs and keys; comma (,) is the default separator.

Important: Alpha-numeric characters are not supported as separators.

Example:

The following batch data request returns the latest stock price, fundamentals, and news for mobile carriers AT&T, T-Mobile, and Verizon.

GET /data/CORE/QUOTE,FUNDAMENTALS,NEWS/T,TMUS,VZ

Result:

The following records are paraphrased for illustration purposes.

[
    {
        "symbol": "T",
        "close": 19.06,
        // more QUOTE record values
    },
    {
        "symbol": "TMUS",
        "close": 145.51,
        // more QUOTE record values
    },
    {
        "symbol": "VZ",
        "close": 38.31,
        // more QUOTE record values
    },
    {
        "symbol": "T",
        "pricePerEarnings": 205.758117055085,
        // more FUNDAMENTALS record values
    },
    {
        "symbol": "TMUS",
        "pricePerEarnings": 46.4663449079113,
        // more FUNDAMENTALS record values
    },
    {
        "symbol": "VZ",
        "pricePerEarnings": 11.6117199849925,
        // more FUNDAMENTALS record values
    },
    {
        "symbol": "T",
        "headline": "Should Investors Buy the Dip in Warner Bros. Discovery Stock?",
        // more NEWS record values
    },
    {
        "symbol": "TMUS",
        "headline": "T-Mobile US, Inc. (TMUS) New Street Research 5G Conference Call (Transcript)",
        // more NEWS record values
    },
    {
        "symbol": "VZ",
        "headline": "Verizon Communications Inc. (VZ) NSR & BCG Innovation Conference: 5G and Beyond. Cloud. Convergence 2022 (Transcript)",
        // more NEWS record values
    },
]

The resulting array includes records for each key (T,TMUS,VZ) found in each dataset (QUOTE,FUNDAMENTALS,NEWS), starting with the first dataset.

The response is an array of dataset records. Records are returned from each dataset for each matched key, in the order that the request specified the datasets and keys. If a key is not matched in a dataset, no record is returned for that key/dataset combination.

If a request’s API token is unauthorized for a dataset, the endpoint returns a standard authorization error.

Important

  • A batch data request must operate on either IEX Cloud Financial Data or your workspace data–the two dataset types can’t be mixed. Batch requests are not supported across workspaces.

  • Each GET /data batch request is limited to 2,000 individual queries. If a batch request exceeds its limit, Apperate reports the error and skips executing the batch request.

Note

  • Record data returned from each dataset counts as a read. See your Apperate plan’s Pricing for reads.

  • Each record returned counts towards your Apperate plan’s maximum records per second. See Pricing for record rate details.

Prerequisite

IEX Cloud Apperate account - Create one here.

Make a Batch Data Request#

Here are steps for making a batch data request.

  1. Determine the key values (primary index values) to query on.

    Example: T,TMUS,VZ for AT&T, T-Mobile, and Verizon.

  2. Determine the IDs of the datasets to search.

    Example: QUOTE,FUNDAMENTALS,NEWS for those IEX Cloud Financial datasets.

    You can browse Workspace Endpoints or the IEX Cloud Data in the API reference or in the Console. (Registeristration required)

  3. Construct your batch data request on the GET /data endpoint, including the dataset IDs, the key values, a batchSeparator parameter (optional), your API token, and any other GET /data query parameters you want.

    Examples:

    /data/CORE/QUOTE,FUNDAMENTALS,NEWS/T,TMUS,VZ?last=2

    /data/MY_WORKSPACE/DATASET_A,DATASET_B/foo,bar,baz?last=2

    /data/CORE/QUOTE.FUNDAMENTALS/SPY.MSFT?batchSeparator=.

  4. Execute your batch request.

    For example, click on the following URL and append your API token as query parameter (e.g., &token=YOUR_TOKEN).

    /data/CORE/QUOTE,FUNDAMENTALS,NEWS/T,TMUS,VZ?last=2

Apperate returns an array of records for each key in each dataset, starting with the first dataset you specified.

Error Reporting#

If your request generates an error, Apperate returns the error as the last object in the response.

Important

Make sure to handle the possibility of the response containing an error message object.

For example, the following response has one record object and an error message object.

[
  {
    "datetime": 1647644242000,
    "hasPaywall": false,
    "headline": "Apple says your next computer isn''t a computer in TV ad for the new iPad Air",
    "image": "https://api.iex.cloud/v1/news/image/ZN4faqJL71CjtjJiUe006uMZuHQLmn5snHo5kWZUUsN",
    "imageUrl": "https://fdn.gsmarena.com/imgroot/news/22/03/ipad-air-ad/-952x498w6/gsmarena_000.jpg",
    "lang": "en",
    "provider": "CityFalcon",
    "qmUrl": "https://www.gsmarena.com/apple_says_your_next_computer_isnt_a_computer_in_tv_ad_for_the_new_ipad_air-news-53630.php?utm_campaign=cityfalcon&utm_medium=cityfalcon&utm_source=cityfalcon",
    "related": "AAPL",
    "source": "GSMArena",
    "summary": "The basic message seems to be: it can do everything.",
    "url": "https://api.iex.cloud/v1/news/article/ZN4faqJL71CjtjJiUe006uMZuHQLmn5snHo5kWZUUsN",
    "id": "NEWS",
    "key": "AAPL",
    "subkey": "ZN4faqJL71CjtjJiUe006uMZuHQLmn5snHo5kWZUUsN",
    "date": 1647644242000,
    "updated": 1647648347000
  },
  {
    "_system_error": "internal error occurred, code 00E866961501B5"
  }
]

Now you know how to conveniently query for multiple keys in multiple datasets.

What’s Next#

Paginate with Chunks of Data using apperate.queryData() in JavaScript or using the GET /data REST endpoint directly. Pagination and handling large amounts of data has never been easier.

Create Views demonstrates combining IEX Cloud Financial Data and workspace data into views.

Querying Data shows how to search data from any dataset (Core or your) using the Get Data endpoint.

Querying Data with iex.js makes querying even easier by leveraging a simple, open-source client library for JavaScript.