Build a Scrapy web spider and set it up on AWS

Închis Postat la acum 6 ani S-au achitat serviciile după ce au fost prestate
Închis S-au achitat serviciile după ce au fost prestate

• Build a web scraper that collects data from the financial times (FT) equity screener website, extracts the relevant data from html and writes this data into a structured cvs-file for download and processing in tabular form by Excel.

• The website URL is as follows: [login to view URL]

• The system should be based on the Scrapy web scraper framework. The spider files written in Python. ([login to view URL])

• The spider should run in fixed intervals, such as once per week. It shall be possible to set and change the frequency later on.

• It should be possible to run several spiders in parallel, each with a specific set of data attributes (e.g. market cap, ROI) collected. For each spider there shall be a specific Python file that can easily be replicated using the code of the initial spider.

• The attributes and target limits (e.g. market cap USD 100M – 1B) are to be set manually. It should be possible to add and delete specific attributes later on as well as change the corresponding target limits. All attributes and target limits are based upon the features of the FT website.

• What might be tricky is that the FT-website uses https. Also the attributes and data ranges cannot be set as parameters in the URL line. When using the website I have to enter the parameters by hand and then submit to get the results list.

• The system shall run on Amazon Web Services (AWS), maybe as an EC2 instance. The files with scraped data shall be stored within a bucket in AWS S3.

• Within the scope of the work shall be the programming of all code required and set up of the live system on AWS for a single initial spider. A login will be provided.

• The scope shall also include a 1-page documentation that describes the structure of the system and gives guidance as to how make the changes described above.

The first spider shall be as follows:

• Interval: Once per week every Thursday

• Website: [login to view URL]

• Attributes for screen and target limits: Countries (Europe – all, America – USA, Canada), Sectors (all), Market cap (USD 500M+), ROI 5 year, ROI current, ROE 5 year, ROE current, Net profit margin 5 year, P/B, P/E, Interest cover, Price change 52 weeks

• Data collected: All columns from the results list, all pages with results sorted alphabetically

Servicii Web Amazon Python Scrapy Arhitectură software Web Scraping

ID Proiect: #16579280

Detalii despre proiect

18 propuneri Proiect la distanță Activ acum 5 ani

18 freelanceri plasează o ofertă medie de 420$ pentru proiect

gangabass

I'm one of the best Scrapy experts here that's why I'm sure you'll be impressed with my work. I can create Python Scrapy based spider(s) (and set it up into your AWS server) that will work exactly like you want. Mai multe

$350 USD în 2 zile
(504 recenzii)
7.5
nmarkovickv

Hi there, I will be happy to build Python script to scrape FT data for you. Feel free to check my profile for reference of my previous work. We are located in the same timezone, you can except prompt response, qui Mai multe

$400 USD în 2 zile
(88 recenzii)
7.0
SigmaVisual

Hi, I have developed similar spiders in scrapy in past. Please let me know if you are interested and I am available to start right away.

$250 USD în 7 zile
(80 recenzii)
7.4
AhmedSalahA

i need more Description and i can do your work don't worry check my profile you will know the price can change after we speak

$400 USD în 5 zile
(64 recenzii)
6.4
hunmin888

hi, employer. i am a python expert. i have a good experience in web scrapping. i have a lot of previous scrapers. so if you award this project to me, i can complete it surely. i wish you will ping me asap. thanks.

$361 USD în 10 zile
(58 recenzii)
6.4
kkc264043kkc

these are my skills set related to web scraping and crawling Have done scraping in Nodejs, CasperJS Phantomjs, python scraping framework Have done testing and automation with selenium also. Know to deal with database Mai multe

$388 USD în 5 zile
(48 recenzii)
6.1
mmadi

Hi , I have good working experience with the required skills & I assure you that I can complete your project "" within the required timeframe.I am keen to work with you. I meet all your requirements. Aso I do ha Mai multe

$280 USD în 12 zile
(11 recenzii)
6.1
shawnwilliams85

Hi I would love to discuss your needs further. I am a full stack developer with 10+years experience and extensive experience building optimized web applications for small to large scale businesses. I strive to off Mai multe

$1558 USD în 10 zile
(6 recenzii)
5.3
logiclast

Hi Dear, I am having an expert level knowledge in website scrapping. I have scrapped 90+ websites for my various customers. Few of the highlights of the websites that I scrapped are LinkedIn, Facebook, Twitter, Insta Mai multe

$388 USD în 10 zile
(4 recenzii)
4.3
vojd11

I checked the site and can assure you that I can develop such scraper. Attributes for screen and target limits can be specified in config file. I can develop this scraper with Scrapy framework but the preferable way fo Mai multe

$350 USD în 10 zile
(10 recenzii)
4.8
arjun366333

Ready to start the work to develop the script for the scrapping to scrap the data from the other website , we can discuss more over chat,thanks regards Arjun S.

$333 USD în 10 zile
(14 recenzii)
4.6
Stonecoldstone

Hello, I find this project interesting and would like to work on it. I have experience building various scraping scripts using python and related modules (BS4, Selenium, Scrapy, lxml, requests). I worked on scrapin Mai multe

$300 USD în 5 zile
(2 recenzii)
2.8
shivangisaini22

Hello, I would like to work with you on this project. For a brief introduction, we are a team ("TECHSON's") of Linux sys admins (RHCSA,LFCS) and system engineers (RHCE,LFCE) and we have 3 years of experience with Li Mai multe

$333 USD în 7 zile
(0 recenzii)
0.0
efibutov

Python experienced developer (5+ yrs) C/C++, VB Databases: MongoDB, PostgreSQL Django/Flask/ReactJS alsoProject Milestone

$361 USD în 10 zile
(0 recenzii)
0.0