Scrapy Jobs

I need a reliable script that pulls fresh product details and current prices from eBay every 24 hours and drops the results into a clean Excel workbook. The data points I absolutely need are: • Full product title • Item ID / listing URL • Current price (and currency) • Shipping cost if shown • Seller name and feedback score • Listing time-stamp so I can track changes day-to-day WORKFLOW OVERVIEW: You will receive: Images from Singapore card vendors showing buyback prices Use AI tools to: Identify the card from the image Search eBay: For recent SOLD listings of the same PSA card Extract: Actual accepted Best Offer prices Calculate: Total landed cost in SGD Compare: Vendor Buyback Price vs Total Cost Flag: Arbitrage opportunity if pro...

We need a Python expert to build an end-to-end automated web-scraping and data-processing system for German B2B directories. The goal is to eliminate all manual steps — scraping, cleaning, deduplication, enrichment, and export — within 15 days. You’ll design and implement a high-speed, fault-tolerant pipeline that scrapes ~10 directories, normalizes data (including Umlauts and formatting), removes duplicates, detects updates, and exports clean structured data automatically. ⸻ Core Responsibilities • Develop asynchronous scraping system (Python 3.11+, aiohttp/httpx/Scrapy/Playwright). • Build deduplication & change-detection logic using hash comparison and timestamps. • Design and connect central database (PostgreSQL + SQLite) to store unique compa...

I have a collection of websites that hold the textual information I need consolidated into a single, well-structured dataset. Rather than copying the material manually, I want the process handled through reliable web-scraping tools so the capture is fast, consistent, and repeatable. Your task is straightforward: • Build (or adapt) a scraper that targets the pages I specify, pulls only the relevant text, and skips ads, navigation links, and other noise. • Deliver the harvested content in a clean CSV or Excel file with clear column headings; if you prefer a database export, let me know and we can adjust. • Include the finished script or notebook so I can rerun the extraction later. Accuracy and formatting matter more to me than sheer speed, so please allow time for basic...

I’ve created a Google Sheet that will act as a central “lead collector,” and I need it filled with fresh, accurate contact data pulled from publicly available sites. The focus is simple yet crucial: for every company you find, capture the homepage URL and a working email address. (ask for details in the sheet ) A completely ethical approach is non-negotiable—no gated content, no third-party lists, and no automated harvesting that violates site terms. I’m happy for you to use tools you’re comfortable with (Python, Scrapy, BeautifulSoup, Selenium, Google Apps Script, etc.) as long as you respect and rate limits. Email addresses must appear in plain text within the sheet; please avoid hyperlinks or HTML encoding. Deliverables • A Google Sheet ...

Je veux constituer un annuaire exhaustif de toutes les bases nautiques en France métropolitaine et DOM-TOM offrant jetski, bouée tractée, flyboard, ski nautique, wakeboard, parachute ascensionnel ou encore location de bateaux. Pour y parvenir, j’ai besoin d’un workflow automatisé reposant exclusivement sur Google Maps – c’est la source retenue – capable de collecter, dédupliquer puis nettoyer les données avant de les mettre en forme dans un CSV directement importable dans le CMS Wix. Les coordonnées devront être fournies en degrés décimaux. Livrables attendus • Un script réutilisable (Python + Selenium, Scrapy ou équivalent) qui interroge Google Maps, gère le r...

Proyecto: API / Sistema de búsqueda de productos en múltiples tiendas (Texto + Foto tipo Lens) Estoy buscando un desarrollador freelance con experiencia en web scraping, desarrollo de APIs y reconocimiento de productos por imagen, para crear un sistema que permita buscar un producto en múltiples tiendas online al mismo tiempo. Objetivo Desarrollar una API (o sistema backend) donde yo pueda buscar productos de dos maneras: 1) Búsqueda por texto Ej: “iPhone 15 Pro 256GB” / “Zapatillas Nike Air Force blancas” 2) Búsqueda por imagen (tipo Google Lens) Subir una foto del producto y que el sistema intente identificarlo (marca/modelo) y luego busque coincidencias en las tiendas. El sistema debe consultar automáticamente una lista...

I need a robust scraping solution that continuously pulls price, full product descriptions, and customer reviews for electronics across roughly ten different e-commerce sites. Each time the script runs, the results should append to a master Excel workbook so that every entry is stored chronologically—allowing me to compare today’s prices with yesterday’s and build long-term trend charts without any manual work. Key expectations • The scraper must visit all target URLs, handle pagination or lazy-loaded content where it exists, and respect each site’s structure. • Collected fields: date/time stamp, site name, product name, current price, full description text, average rating, review count, and a link to the product page. • Excel output: one sheet...

I need an end-to-end AI agent that automatically scouts freelancing websites, general job boards, and specialised training platforms for roles or courses that involve artificial-intelligence work. The agent must: • Crawl and scrape the relevant pages in real time or on a frequent schedule. • Apply NLP or other classification techniques to decide whether a posting is truly AI-related, then tag it by sub-domain (e.g. vision, NLP, MLOps, prompt-engineering). • Deliver concise, deduplicated listings to me through an in-app notification feed—no email or SMS required. For the deployment side I’m open to Python (Scrapy, BeautifulSoup, Selenium), Node, or any stack you are comfortable with so long as it is containerised and can run unattended on a small cloud insta...

Hey! I’m looking to hire an experienced developer to build a universal product-detail scraping pipeline that takes a product URL (any website) and returns a complete structured product record. This is not a “simple HTML parse.” Many target sites are React/Next/Vue, load content via XHR/GraphQL, hide details behind tabs/accordions/modals, and lazy-load images/PDFs. The solution needs to reliably extract everything a human can see on the page, plus the underlying data used to render it. What the scraper must do (high level) Given a product URL, the pipeline should: Load the page like a real user (handle cookies/overlays). Capture all content from multiple sources (DOM + network + interactions). Use GPT API strategically to increase accuracy (field mapping, variant ext...

I need every product on copied into my existing WooCommerce shop so the catalog mirrors theirs one-to-one. That means grabbing each item’s title, plain-text description and all associated images, then pushing them into WordPress with the correct categories, colour swatches and size variations exactly as they appear on Furnx. Only product details are required right now—reviews and live stock counts can wait for a later phase—so the job is focused on clean data capture and a flawless import workflow. Descriptions must remain in plain text; no extra HTML markup. Images should arrive attached to the right variation, including separate gallery shots where available, and the colour options need to show as clickable swatches in WooCommerce, not just text labels. I’m...

I have two source spreadsheets that I need merged and enriched through automated scraping: • “File 1” – 170 k Spanish local businesses with emails • “File 2” – 65 k additional businesses with websites only Phase 1 – Email extraction Using a Python script and well-known libraries (requests, BeautifulSoup, Scrapy or similar), scan every site listed in File 2, capture all working email addresses you can locate, then append them to the corresponding rows so I can produce a unified “File 3”. Phase 2 – Offer harvesting Next, visit each live site in File 3. Where an offer, deal or promotion is publicly displayed, record the details in a fresh Excel sheet with these exact columns: Business ID | Business Name | Offer...

Project Description: Find school districts and charter schools who use a specific vendor for a large list of domains. I am seeking an experienced web scraping specialist to improve our Python script to analyze a large list of school district websites (approximately 4000+ URLs) and identify the ones who show a specific link on any page found in their sitemap. The primary method of identification must be to scan the website's for specific, known vendor links. Deliverables Required 1. A Production-Ready Python Script (.py file): The script must be commented, easily configurable, and capable of reading the provided CSV list, performing the scan, and generating the output CSV. It should handle timeouts and basic error handling gracefully. 2. The Final Results (CSV/Excel File): A c...

I'm looking for an experienced Python developer to create an algorithmic trading bot based on my own unique strategy for options trading. The bot should incorporate the following key functionalities: - Backtesting: Ability to test the strategy against historical data. - Real-time data feed: Integrate with reliable data sources for live market updates. - Automatic order execution: Execute trades automatically based on the strategy signals. Ideal Skills and Experience: - Proficiency in Python and experience with trading libraries. - Strong understanding of options trading and market dynamics. - Experience in developing and backtesting trading algorithms. - Familiarity with integrating real-time data feeds and order execution systems. Please provide examples of previous work and relev...