Simple Scraper/Parser for 10M Queries (High Performance)

$30-250 USD

Finalizat

Data postării:

aproape 4 ani în urmă

$30-250 USD

Plata la predare

Hi, I need a simple parser that is able to make millions of queries in a reasonable time (couple days) to a public API. It needs to support proxies, as the API is very volume-restricted on an IP basis. The parameter passed to the API is a keyword (a company name) and the return object is a small (ca 1kb) XML. Important: You need to parallelize requests / support concurrent queries. This is the main challenge. Save all results to a text file. I have some working code (in Java), however, its performance is not very good. I can give you the following: - sample list of 10.000 keywords/companies - the API endpoint - my existing code (if you like) You can choose any language you like, does not have to be Java. Roughly aiming for 3-5 days turnaround, fast payment. Thanks

ID-ul proiectului: 25774911

Despre proiect

14 propuneri

Proiect la distanță

Activ: 4 ani în urmă

Vrei să câștigi bani?

Adresa de e-mail

Avantajele de a licita pe platforma Freelancer

Stabilește bugetul și intervalul temporal

Îți primești plata pentru serviciile prestate

Evidențiază-ți propunerea

Te înregistrezi și licitezi gratuit pentru proiecte

Acordat utilizatorului:

@elbruninh

⭐⭐⭐Hi, i'm interested, could you give me more details please? do you have a proxy service? can you send me the info? keywords list, API endpoint and your code i can do it with Java (multi-threads) let me know if you are interested [login to view URL]

$160 USD în 3 zile

4,9

(76 recenzii)

6,1

14 freelanceri plasează o ofertă medie de $208 USD pentru proiect

@ramzitra

Hi, I am interested in your project related to build a highly performant scraper to do 10 M queries. Please send me a message so that we can discuss all the details. Thanks, Ramzi

$300 USD în 10 zile

5,0

(534 recenzii)

8,6

@mhmhz

Hi I used to create tool that support multi-threading, proxies. Waiting for discussing to estimate time/feee Thanks

$400 USD în 3 zile

5,0

(130 recenzii)

7,8

@fatimatuzzohura

Hi, I am a professional software developer with 5 years experiences of Python . I have all required skills and knowledge to successfully complete assigned tasks and provide an optimal solution. I am skilled in both back-end and front-end development as well as setting up multi-server environments. Thanks

$250 USD în 5 zile

4,9

(136 recenzii)

6,8

@nisharapath

Hello, how are you? Your description was gone with me well and it is clear for me. My python skill will be applied for this job. Please feel free to contact me. Regards.

$150 USD în 1 zi

5,0

(81 recenzii)

6,6

@benliao

Hello, sir. How are you? I have just read your project description and I can help you as my rich experiences. I have rich experiences in Web Scraping and python I am a new freelancer, but I can give you the perfect result you are satisfied with. Thanks and Regards

$140 USD în 3 zile

4,9

(11 recenzii)

5,1

@AIPDTech

Hi there, We reviewed your requirements and our Fullstack Developers Team with SKILLS - PHP, JAVASCRIPT, Python. We would like to work on web scrapping as per needs and willing to discuss scrapping requirements in detail. We are a reputed IT company with experienced professionals and have been web scrapping for websites from 7+ years. Warm Regards, Naveen

$175 USD în 7 zile

4,9

(8 recenzii)

5,4

@ozo42

hi. is it a public or private api? also how many proxies are available and how many queries can be dobe until they ban the ip?

$247 USD în 2 zile

5,0

(9 recenzii)

4,7

@dduraipandian

I have very good experience in web scraping. My recent scraping job was to scrape where i have to scrape product makers, engine family and product. Difficulties are, 1. search page is a form with javascript event changes. It is not a direct form submission. 2. Once you get to main pages, you will have pagination. Without pagination there are 467 urls to scrape. With pagination, 670 urls to scrape. 3. It is not easy to scrape at once as i get blocked. I scraped with all data with timeouts (without proxies) in one hour (development time is not included). Above can be optimized further with async calls and proxy implementation. Please let me know if you are interested to discuss.

$250 USD în 7 zile

5,0

(11 recenzii)

3,8

@panhavoan

Hello there, I could help you with that project. I've worked with nodejs for years. Puppeteer, x-ray and other supportive are the library that I normally use to scrape data. I have rich experience with this technology. I have completed many scraping projects. I'm a fast worker. I'll deliver the project on time. So please consider hiring me. You can check my recent work with scrapping project. Please contact me I'll send some of my previous project and we could discuss more about this. Thanks

$150 USD în 3 zile

5,0

(8 recenzii)

3,2

@SiniyaBoroda

***Delivery within a few days*** Hello. How are you? I have read your project descriptions in details. I am so interesting in your project as a professional expert in python scraping and building the rpa tools for getting datas by using selenium & bs4. I can start work right now if you want. And I'm sure that I can implement your project perfectly in time as I have already so many project whichever I have done. Looking forward to be your a good venture along with your project. Best Regards.

$200 USD în 3 zile

4,7

(6 recenzii)

3,1

@karmakarsantosh2

Greetings, Hope you are feeling very well. I think here you are looking for someone to build smart file in excel. I understand your project description. But I want to know more about your project. So, can we discuss more? I am really confident that, I can do this task very effectively. I have some good samples and references readily available for a similar kind of project. I'm ready to start now! So, if you hire me then I can be trying to complete this task within your timeline and 100% accurate. Please trust me, I’ll provide you the best quality work. So, please take your time to discuss more on your project with me.

$140 USD în 7 zile

0,0

(0 recenzii)

0,0

@amosso

Hi, I like your project. I have experience in doing parallel processing in Python handling large amounts of data. I can do this in 5 working days. Just share with me your current Java project and some input data for testing and I will deliver an awesome fast API. One question, Which database are you using? Regards. Alex Mosso

$190 USD în 5 zile