Cloudflare::Radar - 来自 Cloudflare Radar 的域名公开数据

Cloudflare Radar 爬虫工具概览
Cloudflare::Radar 爬虫工具从 Cloudflare Radar Web 界面获取有关域名的公开信息:域名卡片、WHOIS、DNS 记录、后量子 TLS 支持、访客国家分布、URL 扫描表和排名历史。
得益于内置强大的 Template Toolkit 模板引擎,您可以按需保存所需格式和结构的结果,该引擎允许对结果应用额外逻辑并以各种格式输出数据,包括 JSON、SQL 和 CSV。
采集数据
$asn— ASN 编号$asn_name— 自治系统名称$asn_country— ASN 国家$dnssec— DNSSEC$created_date,$updated_date,$expiration_date— WHOIS 日期$registrar— 注册商$pq— 后量子 TLS 支持$kexName— 密钥交换名称 (KEX)$categories数组:$categories.$i.name— 网站类别名称
$nameservers数组:$nameservers.$i.name— NS 服务器
$status数组:$status.$i.status— WHOIS 中的域名状态
$records数组:$records.$i.type— DNS 记录类型$records.$i.name— 名称$records.$i.data— 值 (RDATA)
$geo数组(访客分布):$geo.$i.code— 国家代码$geo.$i.value— 来自 Radar 的数值
$scans数组(域名的 URL 扫描):$scans.$i.uuid— UUID$scans.$i.url— URL$scans.$i.time— 时间$scans.$i.isMalicious— 恶意标记
$rankHistory数组:$rankHistory.$i.date— 日期$rankHistory.$i.rank— 排名
应用场景
- 按域名确定网站类别
- ASN 和 DNSSEC 摘要
- WHOIS:日期、注册商、NS、状态
- DNS 记录列表
- 检查后量子 TLS 和 KEX
- 访客地理分布、URL 扫描、域名排名历史
查询
查询时需要提供域名列表,例如:
a-parser.com
yandex.ru
google.com
vk.com
facebook.com
youtube.com
结果输出示例
A-Parser 支持灵活的结果格式化,这得益于内置的 Template Toolkit 模板引擎,使其能够以任意形式以及结构化形式(如 CSV 或 JSON)输出结果。
默认输出
结果格式:
$query: $categories.format('$name, ')\n
包含类别及其描述的结果示例:
a-parser.com: Business, Business & Economy,
yandex.ru: News & Media, Entertainment,
vk.com: Social Networks, Society & Lifestyle,
youtube.com: Video Streaming, Entertainment,
facebook.com: Social Networks, Society & Lifestyle,
google.com: Search Engines, Technology,
输出到 CSV 表格
结果格式:
[% FOREACH categories;
tools.CSVline(name, desc);
END %]
结果示例:
Business,"Sites related to business."
"Business & Economy","Sites that are related to business, economy, finance, education, science and technology."
"Social Networks","Sites that facilitate interaction and networking between people."
"Society & Lifestyle","Sites related to lifestyle that are not included in other categories like fashion, food & drink etc."
"Social Networks","Sites that facilitate interaction and networking between people."
"Society & Lifestyle","Sites related to lifestyle that are not included in other categories like fashion, food & drink etc."
"Search Engines","Sites that allow users to search for content using keywords."
Technology,"Sites related to technology that are not included in the science category."
"News & Media","Sites related to news and media."
Entertainment,"Sites related to entertainment that are not includeded in other categories like Comic books, Audio streaming, Video streaming etc."
将结果转储为 JSON
通用结果格式:
[% IF notFirst;
",\n";
ELSE;
notFirst = 1;
END;
obj = {};
obj.query = query;
obj.categories = [];
FOREACH item IN p1.categories;
obj.categories.push({
name = item.name
desc = item.desc
});
END;
obj.json %]
起始文本:
[
结束文本:
]
结果示例:
[{"query":"yandex.ru","categories":[{"desc":"Sites related to news and media.","name":"News & Media"},{"desc":"Sites related to entertainment that are not includeded in other categories like Comic books, Audio streaming, Video streaming etc.","name":"Entertainment"}]},{"query":"google.com","categories":[{"desc":"Sites that allow users to search for content using keywords.","name":"Search Engines"},{"desc":"Sites related to technology that are not included in the science category.","name":"Technology"}]},{"query":"a-parser.com","categories":[{"desc":"Sites related to business.","name":"Business"},{"desc":"Sites that are related to business, economy, finance, education, science and technology.","name":"Business & Economy"}]}]
提示
要在任务编辑器中使用“Prepend text”和“Append text”选项,需要激活“More options”。
可选设置
| 参数名称 | 默认值 | 描述 |
|---|---|---|
| Bypass CloudFlare | ☑ | 自动绕过 CloudFlare 验证 |
| Bypass CloudFlare with Chrome Max Pages | 10 | 通过 Chrome 绕过 CF 时的最大页面数 |
| Bypass CloudFlare with Chrome Headless | ☑ | 如果启用此选项,通过 Chrome 绕过 CF 时将不显示浏览器界面 |
| Use session | ☑ | 保存良好的会话,从而能够以更少的错误实现更快速的数据抓取。 |