Back to all repositories
GU
scrapy-consistency
by Guidogl
AI agent skill for Scrapy web crawling. Spider vs CrawlSpider patterns, XPath/CSS selectors, item pipelines and loaders, handling pagination and login flows, AUTOTHROTTLE and politeness settings, async/Twisted gotchas, middleware for proxies and user agents, and exporting clean structured data.
0stars0forks0watchers/subscribers0issues
- License
- Apache License 2.0
- Size
- 13 KB
- Created
- Jun 12, 2026
- Last Updated
- Jun 12, 2026
- Last Pushed
- Jun 12, 2026
Available Plugins
Loading plugins...
Evaluate before installing
- Review the source repository, recent maintenance, and license on GitHub.
- Read the marketplace manifest and plugin source files before running commands.
- Start with the smallest required permission set and validate behavior in a safe environment.