SE::Wikipedia::Article - Wikipedia 文章爬虫工具
Wikipedia 爬虫工具概览
Wikipedia 文章爬虫工具。输入文章链接（或短格式 lang:title），输出文章的纯文本全文、简短描述、规范链接、缩略图和原始图片、分类、语言版本、Wikidata ID 以及修订元数据。
数据通过官方 MediaWiki Action API (prop=extracts|info|pageimages|...) 获取。语言根据 URL 或查询前缀确定。Wikipedia 重定向会自动处理（例如：USA → United States）。
如需按关键词搜索文章，请使用 SE::Wikipedia 爬虫工具。
A-Parser 的功能允许保存数据抓取设置以供后续使用（预设），设置数据抓取计划等等。得益于内置强大的模板引擎 Template Toolkit，可以按您需要的形式和结构保存结果。
采集的数据
数据通过 API https://{lang}.wikipedia.org/w/api.php 采集。
- 文章标题 (
title) - 简短描述 (
description) - 文章纯文本全文 (
text) - 规范链接 (
link) - 图片预览图 (
thumb) - 原始图片 (
image) - Page ID (
pageid) - 文章字节大小 (
size) - 出版语言 (
lang) - Wikidata ID (
wikidata) - 最后修订时间 (
timestamp) - 修订 ID (
revid) - 分类列表 (
categories) - 文章的语言版本 (
langlinks: lang, title, link) - API 响应的原始 JSON 数据 (
data)
功能
- 解析 Wikipedia 的桌面版、移动版和
index.php链接 - 支持
lang:title短格式查询 - 自动跟随重定向
- 获取完整的文章纯文本（不含 HTML/Markdown）
- 分类和语言链接支持 API 的
max限制（最多 500 个） - 支持代理和 A-Parser 通用设置
应用场景
- 为 NLP、RAG 和知识库导出文章文本
- 为卡片和机器人采集描述与图片
- 获取页面的分类和语言版本
- 使用文章全文丰富 SE::Wikipedia 的输出结果
查询
支持的查询格式：
- 标准链接：
https://en.wikipedia.org/wiki/Albert_Einstein
https://ru.wikipedia.org/wiki/Ефремов,_Иван_Антонович
- 移动版链接：
https://en.m.wikipedia.org/wiki/Albert_Einstein
index.php形式链接：
https://en.wikipedia.org/w/index.php?title=Albert_Einstein
- 带有章节锚点的链接（
#...会被忽略）：
https://en.wikipedia.org/wiki/Albert_Einstein#Scientific_career
- 短格式
lang:title：
en:Albert Einstein
ru:伊万·安东诺维奇·叶夫列莫夫
eo:Esperanto
同时能正确处理 URL 编码、标题中的 _、重定向以及带连字符的语言代码（如 be-tarask）。
结果输出示例
得益于内置的 Template Toolkit 模板引擎，A-Parser 支持灵活的结果格式化，允许以任意形式输出结果，包括 CSV 或 JSON 等结构化格式。
默认输出
结果格式：
$link\n$title\n$description\n$text\n
结果示例：
https://en.wikipedia.org/wiki/Albert_Einstein
Albert Einstein
German-born theoretical physicist (1879–1955)
Albert Einstein (14 March 1879 – 18 April 1955) was a German-born theoretical physicist best known for developing the theory of relativity. Einstein also made important contributions to quantum theory. His mass–energy equivalence formula E = mc2, which arises from special relativity, has been called "the world's most famous equation". He received the 1921 Nobel Prize in Physics for "his services to theoretical physics, and especially for his discovery of the law of the photoelectric effect".
Born as a subject to the Kingdom of Württemberg, part of the German Empire, Einstein moved to Switzerland in 1895, forsaking his citizenship the following year. In 1896, at the age of seventeen, he enrolled in the mathematics and physics teaching diploma program at the Swiss federal polytechnic school in Zurich, graduating in 1900. He acquired Swiss citizenship a year later, which he kept for the rest of his life, and afterwards secured a permanent position at the Swiss Patent Office in Bern. In 1905, he submitted a successful PhD dissertation to the University of Zurich. In 1914, he moved to Berlin to join the Prussian Academy of Sciences and the Humboldt University of Berlin, becoming director of the Kaiser Wilhelm Institute for Physics in 1917; he also became a Prussian and consequently also German citizen again. In 1933, while Einstein was visiting the United States, Adolf Hitler came to power in Germany. Horrified by the Nazi persecution of his fellow Jews, he decided to remain in the US, and was granted American citizenship in 1940. On the eve of World War II, he endorsed a letter to President Franklin D. Roosevelt alerting him to the potential German nuclear weapons program and recommending that the US begin similar research, later carried out as the Manhattan Project.
...
标题 + 描述 + 链接
结果格式：
$title\t$description\t$link\n
结果示例：
Albert Einstein German-born theoretical physicist (1879–1955) https://en.wikipedia.org/wiki/Albert_Einstein
输出至 CSV
结果格式：
[% tools.CSVline(title, description, link, pageid, wikidata, lang, size) %]
初始文本：
标题,描述,链接,页面 ID,Wikidata,语言,大小
结果示例：
"Albert Einstein","German-born theoretical physicist (1879–1955)",https://en.wikipedia.org/wiki/Albert_Einstein,736,Q937,en,238698
分类
结果格式：
$categories.format('$name\n')
结果示例：
1879 births
1955 deaths
19th-century American Jews
19th-century German Jews
19th-century Swiss Jews
20th-century American engineers
20th-century American inventors
20th-century American letter writers
20th-century American male writers
20th-century American physicists
语言版本
结果格式：
$langlinks.format('$lang\t$title\t$link\n')
结果示例：
ab 阿尔伯特·爱因斯坦 https://ab.wikipedia.org/wiki/%D0%95%D0%B8%D0%BD%D1%88%D1%82%D0%B5%D0%B8%D0%BD,_%D0%90%D0%BB%D0%B1%D0%B5%D1%80%D1%82
af Albert Einstein https://af.wikipedia.org/wiki/Albert_Einstein
gsw Albert Einstein https://als.wikipedia.org/wiki/Albert_Einstein
am አልበርት አይንስታይን https://am.wikipedia.org/wiki/%E1%8A%A0%E1%88%8D%E1%89%A0%E1%88%AD%E1%89%B5_%E1%8A%A0%E1%8B%AD%E1%8A%95%E1%88%B5%E1%89%B3%E1%8B%AD%E1%8A%95
de Albert Einstein https://de.wikipedia.org/wiki/Albert_Einstein
ru 阿尔伯特·爱因斯坦 https://ru.wikipedia.org/wiki/%D0%AD%D0%B9%D0%BD%D1%88%D1%82%D0%B5%D0%B9%D0%BD,_%D0%90%D0%BB%D1%8C%D0%B1%D0%B5%D1%80%D1%82
导出为 JSON
结果格式：
[%
obj = {};
obj.title = title;
obj.description = description;
obj.link = link;
obj.pageid = pageid;
obj.wikidata = wikidata;
obj.lang = lang;
obj.json _ "\n"
%]
结果示例：
{"wikidata":"Q937","link":"https://en.wikipedia.org/wiki/Albert_Einstein","lang":"en","pageid":"736","title":"Albert Einstein","description":"German-born theoretical physicist (1879–1955)"}
可能的设置
该爬虫工具在 UI 中没有独特选项：语言和标题均从查询中获取。仅使用通用设置（代理、重试次数等）。