
Completed
Posted
Paid on delivery
Project Overview We are looking for an experienced PHP developer to create a website scraper for Botanico.ca. The scraper must log in using authorized account credentials, navigate through all product categories, collect product information, download product images, and organize the data into structured product and options tables. Important: Login credentials will be provided privately to the selected developer and must not be included in any public job posting. Required Functionality The scraper should: Log in to the website using the provided credentials. Navigate through all available product categories. Scrape every product and its available details. Identify products that have options, such as size or color. Download up to five images per product. Save images using the SKU-based naming format. Store the scraped information in a structured format, such as MySQL, CSV, or another agreed format. Avoid creating duplicate products when the scraper is run again. Handle pagination and products spread across multiple category pages. Provide clear error handling and logging for failed products or images. Product Table The main product table should contain the following fields: Field Description category Product category or category path title Product title price Product price info Additional product information, if available sku Product SKU size Product size, if applicable color Product color, if applicable stock_qty Available stock quantity, where displayed image1 Filename or path of the first product image image2 Filename or path of the second product image image3 Filename or path of the third product image image4 Filename or path of the fourth product image image5 Filename or path of the fifth product image If no additional product information is available, the info field should remain blank. For example, if a product such as SKU 06-8017 contains the text: SOLD BY CASE QTY ONLY That text should be saved in the info field. Image Requirements Product images must be downloaded into a designated directory. Images should be named according to the following format: text [login to view URL] [login to view URL] [login to view URL] [login to view URL] [login to view URL] For example: text [login to view URL] [login to view URL] The scraper should download a maximum of five images per listing. If a product has fewer than five images, only the available images should be saved. The system should support common image formats, including JPG, JPEG, PNG, and WebP, where applicable. Product Options Table Some products have selectable options, such as size or color. These options should be stored separately and linked to the parent product. The options table should contain: Field Description parent_sku SKU of the main or parent product option_type Option category, such as size or color option_value Specific option value, such as Small, Medium, Large, or Red Example: parent_sku option_type option_value 03-140 size Small 03-140 size Medium 03-140 size Large The scraper should support products with multiple option types, such as both size and color. Expected Deliverables The selected developer will be responsible for providing: Fully functional PHP scraping script. Login and authentication functionality. Complete category and product scraping. Product and options data tables or export files. Downloaded and properly named product images. Duplicate prevention or update functionality. Error logging for failed products, images, or pages. Configuration instructions. Requirements for running the scraper on a server or local computer. Brief documentation explaining how to run the scraper and update its settings. Developer Requirements Applicants should have experience with: PHP. Web scraping and HTML parsing. Login-protected websites. JavaScript-rendered product pages, if applicable. MySQL and/or CSV data exports. Downloading and organizing remote images. Handling pagination and product variations. Session cookies, authentication, and request headers. Writing reliable scrapers that can be rerun without creating duplicate records. Experience with e-commerce websites, product catalogs, inventory data, or marketplace integrations would be an advantage. Please Note if you bid changes or you request for a different amount I will not hire you and you will not be eligible for this project.
Project ID: 40682091
263 proposals
Remote project
Active 5 days ago
Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
263 freelancers are bidding on average $145 USD for this job

With 5+ years of PHP development experience, I understand the need to create a website scraper for Botanico.ca. I have successfully developed similar scrapers in the past for e-commerce platforms. Could you please clarify if the scraper needs to handle dynamic content loading for products on the website? Regards, Yogesh Kumar
$100 USD in 10 days
8.5
8.5

With over a decade of professional experience under my belt, I am uniquely equipped to tackle your web scraping project for Botanico.ca. As a veteran PHP developer and expert in HTML parsing and web scraping, I have an in-depth understanding of how to navigate login-protected websites, handle JavaScript-rendered product pages, and organize scraped data into structured formats like MySQL or CSV. Moreover, my experience in managing remote images effectively aligns with the project's requirement of downloading and correctly naming product images. One area where I particularly excel is developing clean, scalable, and maintainable code that allows for easy updates and minimizes errors. This is vital when creating a scraper of this complexity as it aids in preventing any duplicate products, handling pagination across multiple category pages, and providing clear error handling and logging for failed products or images. In addition to delivering well-commented code, I always provide comprehensive documentation explaining how to run the program and update its settings.
$250 USD in 7 days
8.2
8.2

Hi! This is something we can definitely handle — the spec is detailed enough that we can get straight to work once one thing is clear. The site is built on a platform that may render product pages partially through JavaScript. Depending on how it handles that, we'd use either a straightforward HTTP + session-cookie approach in PHP or route through a headless browser layer to capture everything reliably. The difference matters because one is significantly lighter to run and maintain, and the other adds complexity. Do you know whether the product detail pages load their content dynamically, or does everything come through on the initial HTML response? If you're not sure, we can verify that before we start. Everything else in the spec is well-defined and familiar ground: session-based login, category traversal with pagination, parent/variant product structure, SKU-named image downloads, duplicate prevention on re-runs, and a clean MySQL schema with the product and options tables you laid out. Error logging and a short runbook for re-running the scraper are included as standard — not extras. We'd tackle authentication and a single category first so you can validate the output format and field mapping before we run it across the full catalog. That way any structural surprises on the site get caught early. Looking forward to locking in that one detail and getting started. Gustavo & the DoTheCode team
$230 USD in 9 days
8.0
8.0

Hi, I've built PHP scrapers that log in with session cookies and headers, walk paginated category pages, and rerun without creating duplicates. On Upwork one client wrote that I consistently deliver solid PHP solutions. PHP Programmer Needed for Various Tasks: matched php, javascript, mysql, html One thing that decides reliability here: are Botanico's product pages JavaScript rendered, or is the data in the HTML? If it's JS heavy I'll drive it with a headless browser so options like size and color get captured correctly, otherwise plain HTTP requests keep it fast. Dedupe I'd key on SKU so a second run updates instead of inserting. I keep your amount as posted. I'll start on a milestone so you only release on delivered, working output. One question: is stock_qty shown per option or per product? Adil
$133.10 USD in 7 days
7.6
7.6

As an experienced Web-Scraping Specialist, your project aligns perfectly with my area of expertise. I have honed my skills in extracting valuable data from numerous websites, even those protected by advanced anti-bot measures like Cloudflare and Incapsula. In fact, no website, no matter how intricate or well-defended, can intimidate me. You can be assured that I'll provide you with clean, structured, and accurate data in your desired format (be it MySQL, CSV, etc.). Moreover, I'm well-versed in all aspects of the task you've laid out. From PHP scraping to JavaScript-rendered product pages, I've tackled them all. Over the years, I have developed foolproof methods to logins and passwords for various websites ensuring that I stay within legal boundaries so you won’t need to worry about any privacy concerns—even with login-protected sites like Botanico.ca. In conclusion, hiring me would ensure timely delivery of a comprehensive package containing an efficient PHP scraping script with fully-functioning functionality for login and authentication as well as a detailed documentation for running the scraper both on a server or local computer. My commitment to impeccable work has earned me a long list of satisfied clients—I will deliver the same quality to your project. So let's get started!
$110 USD in 5 days
7.3
7.3

Hello!! I understand you need a PHP developer to improve your existing website and product functionality, fix issues, and make sure everything works reliably without affecting the current features. * Do you already have the complete PHP source code and database access? * Which product features or pages need changes? * Are there any current errors or issues that need immediate attention? The work will include reviewing the existing PHP code, updating product functionality, fixing bugs, improving database interactions, handling forms and validations, and testing the affected workflows across desktop and mobile. Relevant PHP and ecommerce website projects have been maintained and enhanced with focused development, careful debugging, and minimal disruption to existing functionality. Let us chat and review the current website so we can address the required changes efficiently. Best regards Farhin B
$110 USD in 10 days
6.8
6.8

Hi there, I read your project "PHP Developer Needed - Website Product Scraper" and it matches what I do with PHP, Web Development. I have delivered similar work before and can start right away. I can complete it in 7 days for 250 USD, revisions included. Can you share any extra details or files so I can confirm the scope? Happy to start today.
$250 USD in 7 days
6.4
6.4

Hello, The main challenge is making the scraper reliable across the entire catalog while preserving relationships between parent products, options, SKUs, images, and stock data. I would structure the crawler around stable product identifiers, handle pagination and category traversal, normalize option data separately, and make repeated runs update existing records instead of creating duplicates. Image handling also needs to account for different source formats and missing images while enforcing the SKU-based naming convention. Error logging should record the product, failed field or image, and reason so incomplete runs can be diagnosed without manually checking the entire catalog. If you can confirm whether the authenticated catalog is server-rendered or relies heavily on JavaScript requests, I can determine the appropriate scraping approach before implementation. Have a nice day
$120 USD in 4 days
6.0
6.0

As an experienced PHP developer with expertise in web scraping, data processing, and MySQL, I understand the importance of creating a robust website scraper for Botanico.ca. With a proven track record in similar projects, I am confident in delivering a high-quality solution that meets your requirements. I propose to develop a PHP scraping script that logs in securely, navigates through product categories, collects detailed product information, and organizes data into structured tables. By implementing error handling, preventing duplicates, and efficiently managing pagination, I will ensure a seamless scraping process. With a focus on delivering reliable results, I aim to provide a fully functional scraper, structured data tables, properly named images, and clear documentation for easy usage. Let's discuss how I can add value to your project further.
$140 USD in 7 days
6.2
6.2

Hi, I can build the PHP scraper with authenticated sessions, category/pagination traversal, product and option extraction, and SKU-based image downloads. I’ll structure the data in MySQL/CSV, prevent duplicates on reruns, and include reliable error logging and clear configuration/run documentation. I have experience with PHP, web scraping, API/data integrations, and database-driven applications. Thanks Anshuman
$120 USD in 5 days
6.4
6.4

Hi, I am a PHP developer with 8 years of rich experience in software development, with a background in web applications, e commerce platforms, data processing, and scalable backend systems. I am familiar with PHP, JavaScript, MySQL, HTML, web scraping, data processing, and API integration. For this product scraper, I can build a reliable PHP scraper that handles authenticated sessions, category pagination, product variations, image downloads, and structured MySQL or CSV exports. I can also add SKU based image naming, duplicate prevention, error logging, and separate option records so the scraper can be safely rerun without creating duplicate products. I'm an individual freelancer and can work on any time zone you want. Please contact me with the best time for you to have a quick chat. Looking forward to discussing more details. Thanks. Emile.
$250 USD in 7 days
6.0
6.0

Hi, I will be able to create the scrapper in no time and the best part is, you dont have to pay anything until work is complete and you are 100% satisfied...
$50 USD in 3 days
5.8
5.8

Login-protected scrapers on sites like Botanico usually fail at session expiry, not the parsing, so I'll build the login layer to detect a dropped session mid-run and re-auth before continuing. I'll deliver the PHP script with category traversal, the products and options tables keyed on SKU for idempotent reruns, and SKU-named image downloads with a failure log. 1) MySQL direct, or CSV export for you to import? 2) Are option combinations priced separately, or does the parent price cover all variants? Looking forward to it. Shayan
$55 USD in 3 days
5.6
5.6

Hello! I would love to develop the scraping solution for Botanico.ca. I have extensive experience building robust PHP scrapers that handle login-protected environments and complex pagination, including the logic required to correctly store SKU-based image files. I truly enjoy working on automation and data extraction projects as they allow for clean, efficient architectural design. My process involves setting up a secure session handler to maintain your credentials, followed by iterative development to ensure the parent-child SKU relationship is captured perfectly in your database. I have successfully completed similar e-commerce catalog extraction tasks in the past and would be happy to share relevant case studies with you via chat once we connect. I am ready to start immediately. Best regards, Dawid.
$220 USD in 4 days
5.2
5.2

I've built authenticated scrapers for e-commerce sites with product variations and image handling in PHP using Guzzle, Symfony DomCrawler, and SimpleHTMLDom. This is right in my wheelhouse. I'll implement a CLI scraper with Symfony Console, maintain sessions via cookies, parse product pages with DomCrawler, extract options into a separate table, download up to five images per product using cURL with retry logic, store data in MySQL with ON DUPLICATE KEY UPDATE for duplicates, and log failures to a file. The script will use a config file for credentials and output paths, support both MySQL and CSV exports, and include basic error recovery for pagination. All image paths and option relationships will match your schema exactly. I can start immediately. Thanks, Andrii.
$175 USD in 2 days
5.3
5.3

Hello! I see you're looking for a PHP developer to build a website scraper. Your goal is to effectively gather product data, which is crucial for your project. I’ve got solid experience with PHP and web scraping, and I understand the nuances of data processing and storage in MySQL. It would be great to clarify the specific website you want to scrape and any particular data points you need. Are there any rate limits or restrictions from the target site we should be aware of? To ensure a smooth process, I suggest starting with a prototype to validate the scraping logic, then expanding to handle all necessary data. This way, we can mitigate risks early on and adjust as needed. Some similar things I've built: a regional booking platform for a tutoring company, an internal CRM for a property agency, and a React Native field-reporting app. Looking forward to collaborating on this!
$200 USD in 2 days
6.0
6.0

Hi, I have 27 years of professional software engineering experience with strong experience in PHP, MySQL, e-commerce data, web scraping, API/session handling, and structured product catalog systems. I have worked on product and inventory systems where SKU-level data, product variations, stock information, and reliable database handling were important, which directly fits the requirements of this scraper. I’ll build the PHP scraper with authenticated sessions, complete category/pagination traversal, product and option extraction, up to five SKU-named images per product, structured MySQL/CSV output, and rerun-safe duplicate/update logic. I’ll also implement robust error logging, handling for JavaScript-rendered content where required, image format support, failed-page/product recovery, and clear configuration/documentation so you can run and maintain the scraper easily. I will keep the implementation focused on your exact specification and fixed bid, with the final scraper designed to be safely rerun whenever the catalog changes without creating duplicate products.
$140 USD in 7 days
5.3
5.3

I’m an experienced PHP developer skilled in web scraping, MySQL, authentication, and e commerce data extraction. I'll do: Build a reliable PHP scraper with login/session handling. Scrape categories, products, options, pricing, stock, and images. Save up to 5 SKU-based images per product. Q: Does the site require CAPTCHA or any additional verification after login? I’ll follow the exact requirements and quoted amount. Regards
$90 USD in 8 days
5.1
5.1

In my many years working as a PHP developer specializing in web scraping, I've tackled projects just like the one you've outlined for Botanico.ca. From login credentials to pagination and storing information in a structured format, leveraging my technical competence in PHP, MySQL, and JavaScript will ensure I deliver the robust system you desire. Equally important, data integrity is critical when it comes to scraping and organizing large volumes of product information and options. My competences extend beyond PHP and MySQL to encompass other vital technologies like HTML/CSS and JavaScript. In an era where more websites rely on JavaScript-rendered pages for security reasons, my understanding of this technology gives me a significant edge over the competition and allows me to overcome scraping challenges that arise from injecting entry points into these dynamic pages. What sets me apart as a programmer goes beyond my skills - but also encompasses my approach to teamwork. I endeavor to foster everlasting partnership by prioritizing your project goals while adding creative insights along the way. My ultimate objective isn't just delivering functional code or products; it's ensuring I meet your business needs at all times through efficient delivery, reliable execution, and long-term client relationships.
$30 USD in 5 days
5.1
5.1

Hi, I can build the PHP scraper to authenticate securely, crawl all categories and pagination, extract products and options, download up to five SKU named images, and store everything in MySQL or CSV. I’ll include duplicate prevention, support for JavaScript rendered content where required, session handling, retries, and clear error logs for failed products or images. The scraper will be rerunnable without creating duplicate records, with configuration and setup documentation included. Best regards Muhammad Saad
$80 USD in 1 day
5.1
5.1

woodbridge, Canada
Payment method verified
Member since Jan 2, 2011
$30-250 USD
$30-250 USD
$10-30 USD
$30-250 USD
$2-8 USD / hour
₹12500-37500 INR
$30-250 USD
$250-750 USD
₹750-1250 INR / hour
$750-1500 USD
€30-250 EUR
₹600-1500 INR
$250-750 USD
$175-350 USD
$30-250 USD
₹12500-37500 INR
₹750-1250 INR / hour
₹3500-5000 INR
₹750-1250 INR / hour
$10-30 AUD
$30-250 USD
$30-250 NZD
₹12500-37500 INR
₹600-1500 INR
₹600-1500 INR