Find Jobs
Hire Freelancers

Fetch web address with python and log results to MySQL database

$10-30 USD

În desfășurare
Data postării: peste 4 ani în urmă

$10-30 USD

Plata la predare
I have a list of URLs I need fetched via python. The script you build will work like this: 1. Create the database and two tables (in MySQL) via the attached script. 2. Create a python script that takes the file path of the attached CSV and loads the CSV into web_address_lookup.address_input. 3. Create a python script that selects rows from the address_input table WHERE needs_lookup = 1. 3. Iterate through each address, and log back to the address_output table: the address_id, log the http response code to the web_address_lookup.address_output.http_response_code, if the response is a redirect (i.e, http 3xx) log the final URL to the web_address_lookup.address_output.redirected_to_url column, log the HTML returned to [login to view URL] column (discard images and such; this column is a text column on purpose, not a blob -- so images should not be saved), and log the time the lookup was performed to web_address_lookup.address_output.lookup_timestamp. 4. UPDATE web_address_lookup.address_input.needs_lookup to zero. As part of the acceptance criteria, your script will utilize appropriate batching. For example, you should probably fetch 1,000 (or a configurable number...) of rows from address_input, iterate through them, write the results for the entire batch to address_output, update the address_input.needs_lookup column for the entire batch, and then take the next 1,000 records from the address_input table. You will deliver to me: the python code that runs the process described above and the MySQL database proving completion of the tasks. Please note, the sample CSV includes 91,663 rows in it. When I run the process, I will add additional rows to it at various times. As such, the CSV import part of the code needs to be included in the final script (if you use pandas...its probably 3 lines of code or something).
ID-ul proiectului: 20854845

Despre proiect

7 propuneri
Proiect la distanță
Activ: 5 ani în urmă

Vrei să câștigi bani?

Avantajele de a licita pe platforma Freelancer

Stabilește bugetul și intervalul temporal
Îți primești plata pentru serviciile prestate
Evidențiază-ți propunerea
Te înregistrezi și licitezi gratuit pentru proiecte
Acordat utilizatorului:
Avatarul utilizatorului
Hi, I have been doing webscraping along with saving it into database from past 2 years. I have gone through the attachments that you have provided and I would like to work with you on this.
$30 USD în 7 zile
5,0 (1 recenzie)
1,5
1,5
7 freelanceri plasează o ofertă medie de $24 USD pentru proiect
Avatarul utilizatorului
Using PHP will be fine for you? Hello sir, i have 10 years of exp, i have read the description and ready to start work right now. ..................................
$20 USD în 1 zi
5,0 (175 recenzii)
6,7
6,7
Avatarul utilizatorului
Hi Nice to meet you. I have enough experience in python script. Below the libraries are I used in past project. selenium, pandas, matplotlib, lxml, beautifulsoup, scipy, and other useful libraries. I have written some automation and scraping, scientific scripts. So I think I can help you in your project. Just let me know if you want start job. Regards. Lian
$30 USD în 1 zi
4,9 (91 recenzii)
6,6
6,6
Avatarul utilizatorului
Hi. How are you? I read your description carefully and am very interesting in your project. I can start right now and can help you. If you hire me,i will do my best for your job and you can get the best results. Looking forward to hearing from you. Thanks.
$20 USD în 7 zile
4,8 (40 recenzii)
5,6
5,6
Avatarul utilizatorului
Hi,i'm a data scientist with a BS degree in computer science i will do your task as fast as i can and i will achieve it exactly as you want,don't worry about any thing contact me for discussion
$20 USD în 7 zile
4,9 (15 recenzii)
3,9
3,9
Avatarul utilizatorului
Hello, Thanks For good job posted. yes i can do this work so please give me this project. yes i'm free now so if you want i can start this work today. thanks!!
$20 USD în 7 zile
0,0 (0 recenzii)
0,0
0,0
Avatarul utilizatorului
Hi This is how i plan to work on the project, day 1 clarify the requirements with you and create a basic working code. Build on it and share it with you. Day 3 address any issues.
$30 USD în 3 zile
0,0 (0 recenzii)
0,0
0,0

Despre client

Steagul UNITED STATES
Austin, United States
5,0
7
Metoda de plată a fost confirmată
Membru din apr. 25, 2017

Verificarea clientului

Mulțumim! Ți-am trimis prin e-mail linkul pe care trebuie să-l accesezi pentru a revendica creditul gratuit.
A apărut o eroare la trimiterea e-mailului. Încearcă din nou.
Utilizatori înregistrați Totalul proiectelor postate
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Se încarcă previzualizarea
S-a oferit permisiunea de depistare a locației.
Ți-a expirat sesiunea pentru conectare sau te-ai deconectat. Conectează-te din nou.