Parse Web of Science (WoS) plaintext or tab-delimited export files into
an sm_corpus object, preserving authors, fields and cited references.
Usage
sm_read_wos(
path,
encoding = "UTF-8",
engine = c("native", "bibliometrix", "auto"),
verbose = TRUE,
call = rlang::caller_env()
)Arguments
- path
Character scalar. Path to a plaintext or tab-delimited export.
- encoding
Character scalar. File encoding (default
"UTF-8").- engine
Character scalar. One of
"native"(built-in parser),"bibliometrix"(delegate tobibliometrix::convert2df()), or"auto"(try bibliometrix first, fall back to native).- verbose
Logical. Print progress messages?
- call
Caller environment for error reporting.
Value
An sm_corpus object.
Implementation
The native parser follows the Web of Science Core Collection export format.
Tab-delimited files have a header containing PT, AU, TI and UT and one
record per line. Cells are unquoted, so quotes in titles are preserved.
Both layouts share field, author and reference-linking rules. Tab-delimited
input always uses the native path with engine = "native" or "auto";
engine = "bibliometrix" rejects it. UTF-8 BOMs are accepted; UTF-16
exports can be read with encoding = "UTF-16LE".
A file whose records all lack a title, accession number and author is an
error. Such records among readable ones are dropped with one warning,
including when verbose = FALSE. Files without any PT record still
return an empty corpus.
Each record begins with PT (publication type) and ends with ER.
Field tags are two uppercase letters followed by a single space.
Continuation lines begin with three spaces. Tags parsed:
AU or AF (authors), TI (title), SO (source), SN and EI (ISSN and
electronic ISSN), J9 and JI (source abbreviations), AB (abstract),
DI (DOI), PY (year), DT (document type), LA (language), TC (times
cited), UT (accession number, kept in provenance), PM (PubMed ID),
C1 (addresses), RP (corresponding authors), CR (cited references),
DE (author keywords),
ID (Keywords Plus), WC (Web of Science categories) and SC (research
areas). Other tags are not read.
Each distinct SO becomes a row of sources, with issn_l from SN
(EI when SN is missing), issn holding SN and EI, and the abbreviations
J9 and JI in the columns abbreviation and iso_abbreviation.
DE, ID, WC and SC become concepts, one row per ;-separated term, in
the vocabularies "author-keywords", "keywords-plus", "wos-category"
and "wos-subject-category" of sm_concept_vocabularies(). Terms keep
the case of the file.
Author names are stored in the AU form, such as "J Smith" for
"Smith, J", and authors$display_name_alternatives keeps the name as
the file writes it, which tells a surname of several words such as
"Garcia Lopez" from the initials.
Both parsers write one authorship row per author and address. The names in the brackets of C1 link authors to addresses: each name belongs to the author with the same full name (from AF), else to the one with the same surname and initials, else to the one with the same surname and first initial, so a short surname such as "He" never takes the address of "Chen". An address that C1 lists without names, as older records do, belongs to every author of the record, so each of them gets a row for it. An address whose names designate none of the authors belongs to the author of a single-author record and else to nobody.
The corresponding authors come from RP, which names them before
"(corresponding author)" ("(reprint author)" in older files), in any
case, with each name and its address on a line of its own or on one
line. The authors these names designate, matched like the names of C1,
get is_corresponding = TRUE on all their rows and the other authors
FALSE. When a name fits several authors equally (namesakes with the
same surname and initials), the one whose C1 addresses include the
address RP gives with the name is meant; if that does not tell them
apart, they get NA. A record without RP, or whose RP designates none
of its authors for certain, gets NA (unknown) for all its authors;
the native parser reports the number of such records when
verbose = TRUE.
The "bibliometrix" engine (also used by "auto" when bibliometrix can
read the file) converts the data frame of bibliometrix::convert2df()
as as_sm_corpus() does: it reads the same tags into sources,
references and concepts, tells authors apart by the full names of AF
and links C1 and RP to the authors by the same rule.
References
Aria, M. & Cuccurullo, C. (2017). bibliometrix: An R-tool for comprehensive science mapping analysis. Journal of Informetrics, 11(4), 959–975. doi:10.1016/j.joi.2017.08.007
Examples
corpus <- sm_read_wos(sm_example_files("example_wos.txt"), verbose = FALSE)
corpus
#>
#> ── <sm_corpus> ─────────────────────────────────────────────────────────────────
#> Works: 3 | Authors: 9 | Institutions: 0
#> Years: 2022 - 2024
#> Sources (journals): 3
#> Embeddings: none
#> Provenance: wos (3)
#> Status: Unlocked (last refreshed: never)
head(corpus$works)
#> # A tibble: 3 × 16
#> work_id doi title abstract year type source_id cited_by_count oa_status
#> <chr> <chr> <chr> <chr> <int> <chr> <chr> <int> <chr>
#> 1 W000000001 10.1… Spat… Spatial… 2023 Arti… S0000000… 42 NA
#> 2 W000000002 10.1… Mach… We deve… 2022 Arti… S0000000… 18 NA
#> 3 W000000003 10.1… Comp… This st… 2024 Revi… S0000000… 7 NA
#> # ℹ 7 more variables: language <chr>, pmid <chr>, arxiv_id <chr>,
#> # openalex_id <chr>, is_retracted <lgl>, retraction_date <date>,
#> # last_refreshed <dttm>