Query the OpenAlex API for scholarly works and
return the results as an sm_corpus.
Supports both free-text search (query) and structured filter syntax
(filter). Uses cursor-based pagination to retrieve up to n_max works.
Set an OpenAlex API key (see "API key and rate limits").
Besides cited_by_count, the works keep OpenAlex's field-weighted
citation impact (fwci), its citation percentile (citation_percentile,
0 to 1, with the flags is_top_10pct and is_top_1pct) and the date of
these values (citations_as_of), and the corpus gets the yearly citation
counts as citations_by_year (see sm_corpus() and
sm_citation_window()). The openalexR engine has no percentile, so its
three percentile columns are NA. To add these fields to a corpus from
another source, use sm_enrich_openalex_impact().
Usage
sm_fetch_openalex(
query = NULL,
filter = NULL,
n_max = 200L,
per_page = 200L,
mailto = Sys.getenv("SCIMAPR_MAILTO"),
api_key = Sys.getenv("OPENALEX_API_KEY"),
engine = c("native", "openalexR", "auto"),
batch_size = 50L,
verbose = TRUE,
call = rlang::caller_env()
)Arguments
- query
Free-text search query passed to the
searchparameter. IfNULL,filtermust be supplied.- filter
A character string using OpenAlex filter syntax, e.g.
"from_publication_date:2020-01-01,type:journal-article".- n_max
Maximum number of works to return (default 200).
- per_page
Number of results per page (max 200).
- mailto
Contact address sent with each request as the
mailtoparameter. Read from theSCIMAPR_MAILTOenvironment variable by default. OpenAlex no longer gives such requests a larger budget (the API key does), and the openalexR engine does not send it.- api_key
OpenAlex API key. Read from the
OPENALEX_API_KEYenvironment variable by default; see "API key and rate limits".- engine
One of
"native"(built-in httr2 client),"openalexR"(use the openalexR package), or"auto"(use openalexR if version 3.1.1 or later is installed, otherwise native).- batch_size
Integer; maximum number of
|-joined values allowed in a single OpenAlex filter clause before the request is automatically split into multiple batched requests. OpenAlex limits the number of values in anORfilter and the overall URL length, so long DOI (or other ID) lists are chunked, fetched per batch, and row-bound with de-duplication. Default50, which stays comfortably under the API's documented limit of 100.- verbose
Print progress messages?
- call
Caller environment for error reporting.
Value
An sm_corpus object. Its provenance records the search
(query, filter, n_max and engine), so that
sm_rebuild_from_cert() can run it again.
API key and rate limits
OpenAlex expects a free API key. Requests without one share one small
daily budget with everyone on the same network address; once it is used
up, OpenAlex answers HTTP 429 (too many requests) until the budget
resets at midnight UTC. Set the key in the
OPENALEX_API_KEY environment variable (for example in ~/.Renviron)
or pass it as api_key. It is sent in the Authorization header,
never in the URL, and it is stored nowhere in the corpus.
On HTTP 429 the function stops at once with an error of class
scimapR_openalex_rate_limit instead of waiting: the message gives the
reset time and the remaining budget that OpenAlex reports (the fields
reset_at and remaining of the condition) and says whether a key was
sent. Only a wait of up to two seconds (OpenAlex also limits requests per
second) is waited out, once. Other failed requests stop with class
scimapR_openalex_failed; both classes inherit from
scimapR_openalex_error. With engine = "openalexR" the
openalexR package (version 3.1.1 or later) sends the requests: it
gets the key too and its own retries are switched off, but it does not
pass on the response headers, so the reset time and the budget are not
known.
Failed requests
Native requests retry transient server and network failures, and short
announced rate-limit waits. Long waits abort immediately. If a later
page fails, the error inherits from scimapR_fetch_incomplete and holds
the earlier parsed works in rlang::last_error()$partial. Their metadata
records fetch_incomplete with the source, received and expected counts,
and failure reason. If those pages cannot be parsed, partial is NULL.
The default remains an error; an incomplete fetch is never returned as
an ordinary successful result.
Examples
# \donttest{
# Queries the OpenAlex web API.
if (requireNamespace("curl", quietly = TRUE) && curl::has_internet()) {
corpus <- try(sm_fetch_openalex(query = "bibliometrics", n_max = 5))
print(corpus)
}
#> ℹ Fetching works from OpenAlex...
#> ℹ Fetched 5 / 5 works.
#> ✔ Parsed 5 OpenAlex works into sm_corpus.
#>
#> ── <sm_corpus> ─────────────────────────────────────────────────────────────────
#> Works: 5 | Authors: 8 | Institutions: 0
#> Years: 2006 - 2021
#> Sources (journals): 5
#> Embeddings: none
#> Provenance: openalex (5)
#> Status: Unlocked (last refreshed: never)
# }