Help me build a web scraper to extract data from [TARGET].
Target: [WHAT DATA AND FROM WHERE]
Language: [PYTHON / NODE / OTHER]
Volume: [ONE-TIME / PERIODIC / REAL-TIME]
Difficulty: [STATIC HTML / JAVASCRIPT RENDERED / LOGIN REQUIRED / API AVAILABLE]
Provide:
1. **Approach Selection**: Beautiful Soup, Scrapy, Playwright, Puppeteer — which for my case
2. **Target Analysis**: How to inspect the page and identify data locations
3. **Basic Scraper**: Starter code for extracting the data
4. **Pagination Handling**: How to scrape across multiple pages
5. **Dynamic Content**: Handling JavaScript-rendered content
6. **Rate Limiting**: Being respectful with request frequency
7. **Data Storage**: Saving scraped data to CSV, JSON, or database
8. **Error Handling**: Dealing with failed requests, changed layouts, and timeouts
9. **Scheduling**: Running the scraper on a schedule
10. **Legal & Ethical**: robots.txt, terms of service, and ethical scraping practices
Help me build a web scraper to extract data from [TARGET].
Target: [WHAT DATA AND FROM WHERE]
Language: [PYTHON / NODE / OTHER]
Volume: [ONE-TIME / PERIODIC / REAL-TIME]
Difficulty: [STATIC HTML / JAVASCRIPT RENDERED / LOGIN REQUIRED / API AVAILABLE]
Provide:
1. **Approach Selection**: Beautiful Soup, Scrapy, Playwright, Puppeteer — which for my case
2. **Target Analysis**: How to inspect the page and identify data locations
3. **Basic Scraper**: Starter code for extracting the data
4. **Pagination Handling**: How to scrape across multiple pages
5. **Dynamic Content**: Handling JavaScript-rendered content
6. **Rate Limiting**: Being respectful with request frequency
7. **Data Storage**: Saving scraped data to CSV, JSON, or database
8. **Error Handling**: Dealing with failed requests, changed layouts, and timeouts
9. **Scheduling**: Running the scraper on a schedule
10. **Legal & Ethical**: robots.txt, terms of service, and ethical scraping practices
How to use this prompt
Copy the prompt above and paste it into ChatGPT, Claude, Gemini, or any AI assistant. Replace the bracketed placeholders with your specific details.
Leave a Review
Tried this prompt? Let us know how it worked — your reviews help other users find the best prompts.