下拉刷新
Repository Details
Shared bynavbar_avatar
repo_avatar
HelloGitHub Rating
10.0
2 ratings
LLM-Friendly Asynchronous Web Crawler Framework
FreeApache-2.0
Claim
Collect
Share
18.6k
Stars
No
Chinese
HTML
Language
Yes
Active
25
Contributors
111
Issues
No
Organization
None
Latest
1k
Forks
Apache-2.0
License
More
This is an asynchronous web crawler framework developed with Python, capable of transforming website data into LLM-friendly output formats such as Markdown, JSON, etc. It is fully open-source and free, greatly simplifying the writing of asynchronous crawlers. Compared to the paid Firecrawl, it has faster crawling speed, supports simultaneous capture of multiple URLs, page screenshot, keyword optimized extraction (based on LLM), and complex multi-page session management.

Comments

Rating:
No comments yet