Repository Details
Shared by
HelloGitHub Rating
10.0
3 ratings
LLM-Friendly Asynchronous Web Crawler Framework
Past 6 days Received 219 stars ✨
Use arrow keys to explore dates, Home and End for the first and last date, Escape to dismiss.| Date | Star | Daily change |
|---|---|---|
| 周日 | 87,101 | — |
| 周一 | 87,148 | +47 |
| 周二 | 87,192 | +44 |
| 周三 | 87,232 | +40 |
| 周四 | 87,279 | +47 |
| 周五 | 87,320 | +41 |
87.3k
Stars
No
Chinese
Python
Language
Yes
Active
104
Contributors
346
Issues
No
Organization
None
Latest
9k
Forks
Apache-2.0
License
This is an asynchronous web crawler framework developed with Python, capable of transforming website data into LLM-friendly output formats such as Markdown, JSON, etc. It is fully open-source and free, greatly simplifying the writing of asynchronous crawlers. Compared to the paid Firecrawl, it has faster crawling speed, supports simultaneous capture of multiple URLs, page screenshot, keyword optimized extraction (based on LLM), and complex multi-page session management.
Comments
Rating:
No comments yet